[Sorry for the test case bomb]
This patchset splits userfaultfd.c into two tests:
- uffd-stress: the "vanilla", old and powerful stress test
- uffd-unit-tests: all the unit tests will be moved here
This is on my todo list for a long time but I never did it for real. The
uffd test is growing into a small and cute monster. I start to notice it's
going harder to maintain such a test and make it useful.
A few issues I found when looking at userfaultfd test:
- We have a bunch of unit tests in userfaultfd.c, but they always need to
be run only after a stress type. No way to not do it.
- We can only run an unit test for one memory type only, if we want to
do a quick smoke test to check regressions, there's no good way. The
best to come currently is "bash ./run_vmtests.sh -t userfaultfd" thanks
to the most recent changes to run_vmtests.sh on tagging. Still, that
needs to run the stress tests always and hard to see what's wrong.
- It's hard to add a new unit test to userfaultfd.c, we don't really know
what's happening, not until we mostly read the whole file.
- We did a bunch of useless tests, e.g. we run twice the whole suite of
stress test just to verify both syscall and /dev/userfaultfd. They're
all using userfaultfd_new() to create the handle, everything should
really be the same underneath. One simple unit test should cover that!
- We have tens of global variables in one file but shared with all the
tests. Some of them are not suitable to be a global var from
maintainance pov. It enforces every unit test to consider how these
vars affects the stress test and vice versa, but that's logically not
necessary.
- Userfaultfd test is not friendly to old kernels. Mostly it only works
on the latest kernel tree. It's preferrable to be run on all kernels
and properly report what's missing.
I'll stop here, I feel like I can still list some..
This patchset should resolve all issues above, and actually we can do even
more on top. I stopped doing that until I found I already got 29 patches
and 2000+ LOC changes. That's already a patchset terrible enough so we
should move in small steps.
After the whole set applied, "./run_vmtests.sh -t userfaultfd" looks like
this:
===8<===
vm.nr_hugepages = 1024
-------------------------
running ./uffd-unit-tests
-------------------------
UFFDIO_API (with syscall)... done
UFFDIO_API (with /dev/userfaultfd)... done
zeropage on anon... done
zeropage on shmem... done
zeropage on shmem-private... done
zeropage-hugetlb on hugetlb... done
zeropage-hugetlb on hugetlb-private... done
pagemap on anon... pagemap on anon... pagemap on anon... done
wp-unpopulated on anon... done
minor on shmem... done
minor on hugetlb... done
minor-wp on shmem... done
minor-wp on hugetlb... done
minor-collapse on shmem... done
sigbus on anon... sigbus on anon... done
sigbus on shmem... sigbus on shmem... done
sigbus on shmem-private... sigbus on shmem-private... done
sigbus on hugetlb... sigbus on hugetlb... done
sigbus on hugetlb-private... sigbus on hugetlb-private... done
sigbus-wp on anon... sigbus-wp on anon... done
sigbus-wp on shmem... sigbus-wp on shmem... done
sigbus-wp on shmem-private... sigbus-wp on shmem-private... done
sigbus-wp on hugetlb... sigbus-wp on hugetlb... done
sigbus-wp on hugetlb-private... sigbus-wp on hugetlb-private... done
events on anon... events on anon... done
events on shmem... events on shmem... done
events on shmem-private... events on shmem-private... done
events on hugetlb... events on hugetlb... done
events on hugetlb-private... events on hugetlb-private... done
events-wp on anon... events-wp on anon... done
events-wp on shmem... events-wp on shmem... done
events-wp on shmem-private... events-wp on shmem-private... done
events-wp on hugetlb... events-wp on hugetlb... done
events-wp on hugetlb-private... events-wp on hugetlb-private... done
Userfaults unit tests: pass=34, skip=0, fail=0 (total=34)
[PASS]
--------------------------------
running ./uffd-stress anon 20 16
--------------------------------
nr_pages: 5120, nr_pages_per_cpu: 640
bounces: 15, mode: rnd racing ver poll, userfaults: 345 missing (26+48+61+102+30+12+59+7) 1596 wp (120+139+317+346+215+67+306+86)
[...]
[PASS]
------------------------------------
running ./uffd-stress hugetlb 128 32
------------------------------------
nr_pages: 64, nr_pages_per_cpu: 8
bounces: 31, mode: rnd racing ver poll, userfaults: 29 missing (6+6+6+5+4+2+0+0) 104 wp (20+19+22+18+7+12+5+1)
[...]
[PASS]
--------------------------------------------
running ./uffd-stress hugetlb-private 128 32
--------------------------------------------
nr_pages: 64, nr_pages_per_cpu: 8
bounces: 31, mode: rnd racing ver poll, userfaults: 33 missing (12+9+7+0+5+0+0+0) 111 wp (24+25+14+14+11+17+5+1)
[...]
[PASS]
---------------------------------
running ./uffd-stress shmem 20 16
---------------------------------
nr_pages: 5120, nr_pages_per_cpu: 640
bounces: 15, mode: rnd racing ver poll, userfaults: 247 missing (15+17+34+60+81+37+3+0) 2038 wp (180+114+276+400+381+318+165+204)
[...]
[PASS]
-----------------------------------------
running ./uffd-stress shmem-private 20 16
-----------------------------------------
nr_pages: 5120, nr_pages_per_cpu: 640
bounces: 15, mode: rnd racing ver poll, userfaults: 235 missing (52+29+55+56+13+9+16+5) 2849 wp (218+406+461+531+328+284+430+191)
[...]
[PASS]
SUMMARY: PASS=6 SKIP=0 FAIL=0
===8<===
The output may be different if we miss some features (e.g., hugetlb not
allocated, old kernel, less privilege of uffd handle), but they should show
up with good reasons. E.g., I tried to run the unit test on my Fedora
kernel and it gives me:
===8<===
UFFDIO_API (with syscall)... failed [reason: UFFDIO_API should fail with wrong api but didn't]
UFFDIO_API (with /dev/userfaultfd)... skipped [reason: cannot open userfaultfd handle]
zeropage on anon... done
zeropage on shmem... done
zeropage on shmem-private... done
zeropage-hugetlb on hugetlb... done
zeropage-hugetlb on hugetlb-private... done
pagemap on anon... pagemap on anon... pagemap on anon... done
wp-unpopulated on anon... skipped [reason: feature missing]
minor on shmem... done
minor on hugetlb... done
minor-wp on shmem... skipped [reason: feature missing]
minor-wp on hugetlb... skipped [reason: feature missing]
minor-collapse on shmem... done
sigbus on anon... skipped [reason: possible lack of priviledge]
sigbus on shmem... skipped [reason: possible lack of priviledge]
sigbus on shmem-private... skipped [reason: possible lack of priviledge]
sigbus on hugetlb... skipped [reason: possible lack of priviledge]
sigbus on hugetlb-private... skipped [reason: possible lack of priviledge]
sigbus-wp on anon... skipped [reason: possible lack of priviledge]
sigbus-wp on shmem... skipped [reason: possible lack of priviledge]
sigbus-wp on shmem-private... skipped [reason: possible lack of priviledge]
sigbus-wp on hugetlb... skipped [reason: possible lack of priviledge]
sigbus-wp on hugetlb-private... skipped [reason: possible lack of priviledge]
events on anon... skipped [reason: possible lack of priviledge]
events on shmem... skipped [reason: possible lack of priviledge]
events on shmem-private... skipped [reason: possible lack of priviledge]
events on hugetlb... skipped [reason: possible lack of priviledge]
events on hugetlb-private... skipped [reason: possible lack of priviledge]
events-wp on anon... skipped [reason: possible lack of priviledge]
events-wp on shmem... skipped [reason: possible lack of priviledge]
events-wp on shmem-private... skipped [reason: possible lack of priviledge]
events-wp on hugetlb... skipped [reason: possible lack of priviledge]
events-wp on hugetlb-private... skipped [reason: possible lack of priviledge]
Userfaults unit tests: pass=9, skip=24, fail=1 (total=34)
===8<===
Patch layout:
- Revert "userfaultfd: don't fail on unrecognized features"
Something I found when I got the UFFDIO_API test below. Axel, I still
propose to revert it as a whole, but feel free to continue the discussion
from the original patch thread.
- selftests/mm: Update .gitignore with two missing tests
- selftests/mm: Dump a summary in run_vmtests.sh
- selftests/mm: Merge util.h into vm_util.h
- selftests/mm: Use TEST_GEN_PROGS where proper
- selftests/mm: Link vm_util.c always
- selftests/mm: Merge default_huge_page_size() into one
- selftests/mm: Use PM_* macros in vm_utils.h
- selftests/mm: Reuse pagemap_get_entry() in vm_util.h
- selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb
- selftests/mm: Drop test_uffdio_zeropage_eexist
Until here, all cleanups here and there. I wanted to keep going, but I
found that maybe it'll take a few more days to split the test. Hence I
did a split starting from the next one, so we have a working thing first.
- selftests/mm: Create uffd-common.[ch]
- selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests
This did the major brute force split of common codes into
uffd-common.[ch]. That'll be the so far common base for stress and unit
tests. Then a new unit test is created.
- selftests/mm: uffd_[un]register()
- selftests/mm: uffd_open_{dev|sys}()
- selftests/mm: UFFDIO_API test
This patch hides here to start writting the 1st unit test with
UFFDIO_API, also detection of userfaultfd privileges.
- selftests/mm: Drop global mem_fd in uffd tests
- selftests/mm: Drop global hpage_size in uffd tests
- selftests/mm: Let uffd_handle_page_fault() takes wp parameter
- selftests/mm: Allow allocate_area() to fail properly
Some further cleanup that I noticed otherwise hard to move the tests.
- selftests/mm: Add framework for uffd-unit-test
The major patch provides the framework for most of the rest unit tests.
- selftests/mm: Move uffd pagemap test to unit test
- selftests/mm: Move uffd minor test to unit test
- selftests/mm: Move uffd sig/events tests into uffd unit tests
- selftests/mm: Move zeropage test into uffd unit tests
Move unit tests and suite them into the new file.
- selftests/mm: Workaround no way to detect uffd-minor + wp
- selftests/mm: Allow uffd test to skip properly with no privilege
- selftests/mm: Drop sys/dev test in uffd-stress test
- selftests/mm: Add shmem-private test to uffd-stress
A bunch of changes to do better on error reportings, and add
shmem-private to the stress test which was long missing.
Please have a look, any comment welcomed.
Thanks,
Peter Xu (29):
Revert "userfaultfd: don't fail on unrecognized features"
selftests/mm: Update .gitignore with two missing tests
selftests/mm: Dump a summary in run_vmtests.sh
selftests/mm: Merge util.h into vm_util.h
selftests/mm: Use TEST_GEN_PROGS where proper
selftests/mm: Link vm_util.c always
selftests/mm: Merge default_huge_page_size() into one
selftests/mm: Use PM_* macros in vm_utils.h
selftests/mm: Reuse pagemap_get_entry() in vm_util.h
selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb
selftests/mm: Drop test_uffdio_zeropage_eexist
selftests/mm: Create uffd-common.[ch]
selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests
selftests/mm: uffd_[un]register()
selftests/mm: uffd_open_{dev|sys}()
selftests/mm: UFFDIO_API test
selftests/mm: Drop global mem_fd in uffd tests
selftests/mm: Drop global hpage_size in uffd tests
selftests/mm: Let uffd_handle_page_fault() takes wp parameter
selftests/mm: Allow allocate_area() to fail properly
selftests/mm: Add framework for uffd-unit-test
selftests/mm: Move uffd pagemap test to unit test
selftests/mm: Move uffd minor test to unit test
selftests/mm: Move uffd sig/events tests into uffd unit tests
selftests/mm: Move zeropage test into uffd unit tests
selftests/mm: Workaround no way to detect uffd-minor + wp
selftests/mm: Allow uffd test to skip properly with no privilege
selftests/mm: Drop sys/dev test in uffd-stress test
selftests/mm: Add shmem-private test to uffd-stress
fs/userfaultfd.c | 6 +-
tools/testing/selftests/mm/.gitignore | 5 +-
tools/testing/selftests/mm/Makefile | 77 +-
tools/testing/selftests/mm/gup_test.c | 5 +-
tools/testing/selftests/mm/hugepage-mremap.c | 7 +-
tools/testing/selftests/mm/hugetlb-madvise.c | 25 +-
.../selftests/mm/ksm_functional_tests.c | 6 +-
tools/testing/selftests/mm/ksm_tests.c | 2 +-
tools/testing/selftests/mm/mrelease_test.c | 11 +-
tools/testing/selftests/mm/run_vmtests.sh | 26 +-
tools/testing/selftests/mm/thuge-gen.c | 19 +-
tools/testing/selftests/mm/transhuge-stress.c | 12 +-
tools/testing/selftests/mm/uffd-common.c | 612 ++++++
tools/testing/selftests/mm/uffd-common.h | 117 +
tools/testing/selftests/mm/uffd-stress.c | 482 +++++
tools/testing/selftests/mm/uffd-unit-tests.c | 891 ++++++++
tools/testing/selftests/mm/userfaultfd.c | 1903 -----------------
tools/testing/selftests/mm/util.h | 69 -
tools/testing/selftests/mm/vm_util.c | 166 +-
tools/testing/selftests/mm/vm_util.h | 48 +
20 files changed, 2385 insertions(+), 2104 deletions(-)
create mode 100644 tools/testing/selftests/mm/uffd-common.c
create mode 100644 tools/testing/selftests/mm/uffd-common.h
create mode 100644 tools/testing/selftests/mm/uffd-stress.c
create mode 100644 tools/testing/selftests/mm/uffd-unit-tests.c
delete mode 100644 tools/testing/selftests/mm/userfaultfd.c
delete mode 100644 tools/testing/selftests/mm/util.h
--
2.39.1
This is a proposal to revert commit 914eedcb9ba0ff53c33808.
I found this when writting a simple UFFDIO_API test to be the first unit
test in this set. Two things breaks with the commit:
- UFFDIO_API check was lost and missing. According to man page, the
kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
check is needed if the api version will be extended in the future, or
user app won't be able to identify which is a new kernel.
- Feature flags checks were removed, which means UFFDIO_API with a
feature that does not exist will also succeed. According to the man
page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
unknown features passed in.
Link: https://lore.kernel.org/r/[email protected]
Cc: Axel Rasmussen <[email protected]>
Cc: linux-stable <[email protected]>
Signed-off-by: Peter Xu <[email protected]>
---
fs/userfaultfd.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 8395605790f6..3b2a41c330e6 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -1977,8 +1977,10 @@ static int userfaultfd_api(struct userfaultfd_ctx *ctx,
ret = -EFAULT;
if (copy_from_user(&uffdio_api, buf, sizeof(uffdio_api)))
goto out;
- /* Ignore unsupported features (userspace built against newer kernel) */
- features = uffdio_api.features & UFFD_API_FEATURES;
+ features = uffdio_api.features;
+ ret = -EINVAL;
+ if (uffdio_api.api != UFFD_API || (features & ~UFFD_API_FEATURES))
+ goto err_out;
ret = -EPERM;
if ((features & UFFD_FEATURE_EVENT_FORK) && !capable(CAP_SYS_PTRACE))
goto err_out;
--
2.39.1
There're two util headers under mm/ kselftest. Merge one with another. It
turns out util.h is the easy one to move.
When merging, drop PAGE_SIZE / PAGE_SHIFT because they're unnecessary
wrappers to page_size() / page_shift(), meanwhile rename them to psize()
and pshift() so as to not conflict with some existing definitions in some
test files that includes vm_util.h.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/Makefile | 4 ++
tools/testing/selftests/mm/gup_test.c | 5 +-
tools/testing/selftests/mm/ksm_tests.c | 2 +-
tools/testing/selftests/mm/mrelease_test.c | 11 ++-
tools/testing/selftests/mm/transhuge-stress.c | 12 ++--
tools/testing/selftests/mm/util.h | 69 -------------------
tools/testing/selftests/mm/vm_util.c | 31 +++++++++
tools/testing/selftests/mm/vm_util.h | 31 +++++++++
8 files changed, 80 insertions(+), 85 deletions(-)
delete mode 100644 tools/testing/selftests/mm/util.h
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index fbf5646b1072..4188435967ed 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -111,6 +111,10 @@ $(OUTPUT)/madv_populate: vm_util.c
$(OUTPUT)/soft-dirty: vm_util.c
$(OUTPUT)/split_huge_page_test: vm_util.c
$(OUTPUT)/userfaultfd: vm_util.c
+$(OUTPUT)/gup_test: vm_util.c
+$(OUTPUT)/mrelease_test: vm_util.c
+$(OUTPUT)/transhuge-stress: vm_util.c
+$(OUTPUT)/ksm_tests: vm_util.c
ifeq ($(MACHINE),x86_64)
BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
diff --git a/tools/testing/selftests/mm/gup_test.c b/tools/testing/selftests/mm/gup_test.c
index e43879291dac..ec2229136384 100644
--- a/tools/testing/selftests/mm/gup_test.c
+++ b/tools/testing/selftests/mm/gup_test.c
@@ -12,8 +12,7 @@
#include <assert.h>
#include <mm/gup_test.h>
#include "../kselftest.h"
-
-#include "util.h"
+#include "vm_util.h"
#define MB (1UL << 20)
@@ -251,7 +250,7 @@ int main(int argc, char **argv)
if (touch) {
gup.gup_flags |= FOLL_TOUCH;
} else {
- for (; (unsigned long)p < gup.addr + size; p += PAGE_SIZE)
+ for (; (unsigned long)p < gup.addr + size; p += psize())
p[0] = 0;
}
diff --git a/tools/testing/selftests/mm/ksm_tests.c b/tools/testing/selftests/mm/ksm_tests.c
index 9fb21b982dc9..85a49aea3ab8 100644
--- a/tools/testing/selftests/mm/ksm_tests.c
+++ b/tools/testing/selftests/mm/ksm_tests.c
@@ -14,7 +14,7 @@
#include "../kselftest.h"
#include <include/vdso/time64.h>
-#include "util.h"
+#include "vm_util.h"
#define KSM_SYSFS_PATH "/sys/kernel/mm/ksm/"
#define KSM_FP(s) (KSM_SYSFS_PATH s)
diff --git a/tools/testing/selftests/mm/mrelease_test.c b/tools/testing/selftests/mm/mrelease_test.c
index 6c62966ab5db..37b6d33b9e84 100644
--- a/tools/testing/selftests/mm/mrelease_test.c
+++ b/tools/testing/selftests/mm/mrelease_test.c
@@ -9,8 +9,7 @@
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
-
-#include "util.h"
+#include "vm_util.h"
#include "../kselftest.h"
@@ -32,7 +31,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
unsigned long i;
char *buf;
- buf = (char *)mmap(NULL, nr_pages * PAGE_SIZE, PROT_READ | PROT_WRITE,
+ buf = (char *)mmap(NULL, nr_pages * psize(), PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANON, 0, 0);
if (buf == MAP_FAILED) {
perror("mmap failed, halting the test");
@@ -40,7 +39,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
}
for (i = 0; i < nr_pages; i++)
- *((unsigned long *)(buf + (i * PAGE_SIZE))) = i;
+ *((unsigned long *)(buf + (i * psize()))) = i;
/* Signal the parent that the child is ready */
if (write(pipefd, "", 1) < 0) {
@@ -54,7 +53,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
timeout--;
}
- munmap(buf, nr_pages * PAGE_SIZE);
+ munmap(buf, nr_pages * psize());
return (timeout > 0) ? KSFT_PASS : KSFT_FAIL;
}
@@ -87,7 +86,7 @@ static int child_main(int pipefd[], size_t size)
/* Allocate and fault-in memory and wait to be killed */
close(pipefd[0]);
- res = alloc_noexit(MB(size) / PAGE_SIZE, pipefd[1]);
+ res = alloc_noexit(MB(size) / psize(), pipefd[1]);
close(pipefd[1]);
return res;
}
diff --git a/tools/testing/selftests/mm/transhuge-stress.c b/tools/testing/selftests/mm/transhuge-stress.c
index e3f00adb1b82..ba9d37ad3a89 100644
--- a/tools/testing/selftests/mm/transhuge-stress.c
+++ b/tools/testing/selftests/mm/transhuge-stress.c
@@ -15,7 +15,7 @@
#include <fcntl.h>
#include <string.h>
#include <sys/mman.h>
-#include "util.h"
+#include "vm_util.h"
int backing_fd = -1;
int mmap_flags = MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE;
@@ -34,10 +34,10 @@ int main(int argc, char **argv)
int pagemap_fd;
ram = sysconf(_SC_PHYS_PAGES);
- if (ram > SIZE_MAX / sysconf(_SC_PAGESIZE) / 4)
+ if (ram > SIZE_MAX / psize() / 4)
ram = SIZE_MAX / 4;
else
- ram *= sysconf(_SC_PAGESIZE);
+ ram *= psize();
len = ram;
while (++i < argc) {
@@ -58,7 +58,7 @@ int main(int argc, char **argv)
warnx("allocate %zd transhuge pages, using %zd MiB virtual memory"
" and %zd MiB of ram", len >> HPAGE_SHIFT, len >> 20,
- ram >> (20 + HPAGE_SHIFT - PAGE_SHIFT - 1));
+ ram >> (20 + HPAGE_SHIFT - pshift() - 1));
pagemap_fd = open("/proc/self/pagemap", O_RDONLY);
if (pagemap_fd < 0)
@@ -92,7 +92,7 @@ int main(int argc, char **argv)
if (pfn < 0) {
nr_failed++;
} else {
- size_t idx = pfn >> (HPAGE_SHIFT - PAGE_SHIFT);
+ size_t idx = pfn >> (HPAGE_SHIFT - pshift());
nr_succeed++;
if (idx >= map_len) {
@@ -108,7 +108,7 @@ int main(int argc, char **argv)
}
/* split transhuge page, keep last page */
- if (madvise(p, HPAGE_SIZE - PAGE_SIZE, MADV_DONTNEED))
+ if (madvise(p, HPAGE_SIZE - psize(), MADV_DONTNEED))
err(2, "MADV_DONTNEED");
}
clock_gettime(CLOCK_MONOTONIC, &b);
diff --git a/tools/testing/selftests/mm/util.h b/tools/testing/selftests/mm/util.h
deleted file mode 100644
index b27d26199334..000000000000
--- a/tools/testing/selftests/mm/util.h
+++ /dev/null
@@ -1,69 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-
-#ifndef __KSELFTEST_VM_UTIL_H
-#define __KSELFTEST_VM_UTIL_H
-
-#include <stdint.h>
-#include <sys/mman.h>
-#include <err.h>
-#include <string.h> /* ffsl() */
-#include <unistd.h> /* _SC_PAGESIZE */
-
-static unsigned int __page_size;
-static unsigned int __page_shift;
-
-static inline unsigned int page_size(void)
-{
- if (!__page_size)
- __page_size = sysconf(_SC_PAGESIZE);
- return __page_size;
-}
-
-static inline unsigned int page_shift(void)
-{
- if (!__page_shift)
- __page_shift = (ffsl(page_size()) - 1);
- return __page_shift;
-}
-
-#define PAGE_SHIFT (page_shift())
-#define PAGE_SIZE (page_size())
-/*
- * On ppc64 this will only work with radix 2M hugepage size
- */
-#define HPAGE_SHIFT 21
-#define HPAGE_SIZE (1 << HPAGE_SHIFT)
-
-#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
-#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
-
-
-static inline int64_t allocate_transhuge(void *ptr, int pagemap_fd)
-{
- uint64_t ent[2];
-
- /* drop pmd */
- if (mmap(ptr, HPAGE_SIZE, PROT_READ | PROT_WRITE,
- MAP_FIXED | MAP_ANONYMOUS |
- MAP_NORESERVE | MAP_PRIVATE, -1, 0) != ptr)
- errx(2, "mmap transhuge");
-
- if (madvise(ptr, HPAGE_SIZE, MADV_HUGEPAGE))
- err(2, "MADV_HUGEPAGE");
-
- /* allocate transparent huge page */
- *(volatile void **)ptr = ptr;
-
- if (pread(pagemap_fd, ent, sizeof(ent),
- (uintptr_t)ptr >> (PAGE_SHIFT - 3)) != sizeof(ent))
- err(2, "read pagemap");
-
- if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
- PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
- !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - PAGE_SHIFT)) - 1)))
- return PAGEMAP_PFN(ent[0]);
-
- return -1;
-}
-
-#endif
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 40e795624ff3..0204c469be43 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -8,6 +8,9 @@
#define SMAP_FILE_PATH "/proc/self/smaps"
#define MAX_LINE_LENGTH 500
+unsigned int __page_size;
+unsigned int __page_shift;
+
uint64_t pagemap_get_entry(int fd, char *start)
{
const unsigned long pfn = (unsigned long)start / getpagesize();
@@ -149,3 +152,31 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size)
{
return __check_huge(addr, "ShmemPmdMapped:", nr_hpages, hpage_size);
}
+
+int64_t allocate_transhuge(void *ptr, int pagemap_fd)
+{
+ uint64_t ent[2];
+
+ /* drop pmd */
+ if (mmap(ptr, HPAGE_SIZE, PROT_READ | PROT_WRITE,
+ MAP_FIXED | MAP_ANONYMOUS |
+ MAP_NORESERVE | MAP_PRIVATE, -1, 0) != ptr)
+ errx(2, "mmap transhuge");
+
+ if (madvise(ptr, HPAGE_SIZE, MADV_HUGEPAGE))
+ err(2, "MADV_HUGEPAGE");
+
+ /* allocate transparent huge page */
+ *(volatile void **)ptr = ptr;
+
+ if (pread(pagemap_fd, ent, sizeof(ent),
+ (uintptr_t)ptr >> (pshift() - 3)) != sizeof(ent))
+ err(2, "read pagemap");
+
+ if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
+ PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
+ !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - pshift())) - 1)))
+ return PAGEMAP_PFN(ent[0]);
+
+ return -1;
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 1995ee911ef2..6edeb531afc6 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -1,6 +1,27 @@
/* SPDX-License-Identifier: GPL-2.0 */
#include <stdint.h>
#include <stdbool.h>
+#include <sys/mman.h>
+#include <err.h>
+#include <string.h> /* ffsl() */
+#include <unistd.h> /* _SC_PAGESIZE */
+
+extern unsigned int __page_size;
+extern unsigned int __page_shift;
+
+static inline unsigned int psize(void)
+{
+ if (!__page_size)
+ __page_size = sysconf(_SC_PAGESIZE);
+ return __page_size;
+}
+
+static inline unsigned int pshift(void)
+{
+ if (!__page_shift)
+ __page_shift = (ffsl(psize()) - 1);
+ return __page_shift;
+}
uint64_t pagemap_get_entry(int fd, char *start);
bool pagemap_is_softdirty(int fd, char *start);
@@ -13,3 +34,13 @@ uint64_t read_pmd_pagesize(void);
bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);
bool check_huge_file(void *addr, int nr_hpages, uint64_t hpage_size);
bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
+int64_t allocate_transhuge(void *ptr, int pagemap_fd);
+
+/*
+ * On ppc64 this will only work with radix 2M hugepage size
+ */
+#define HPAGE_SHIFT 21
+#define HPAGE_SIZE (1 << HPAGE_SHIFT)
+
+#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
+#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
--
2.39.1
There're already 3 same definitions of the three functions. Move it into
vm_util.[ch].
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/hugetlb-madvise.c | 25 +-------------------
tools/testing/selftests/mm/thuge-gen.c | 19 +--------------
tools/testing/selftests/mm/userfaultfd.c | 24 -------------------
tools/testing/selftests/mm/vm_util.c | 21 ++++++++++++++++
tools/testing/selftests/mm/vm_util.h | 1 +
5 files changed, 24 insertions(+), 66 deletions(-)
diff --git a/tools/testing/selftests/mm/hugetlb-madvise.c b/tools/testing/selftests/mm/hugetlb-madvise.c
index 9a127a8fe176..28426e30d9bc 100644
--- a/tools/testing/selftests/mm/hugetlb-madvise.c
+++ b/tools/testing/selftests/mm/hugetlb-madvise.c
@@ -18,6 +18,7 @@
#include <unistd.h>
#include <sys/mman.h>
#include <fcntl.h>
+#include "vm_util.h"
#define MIN_FREE_PAGES 20
#define NR_HUGE_PAGES 10 /* common number of pages to map/allocate */
@@ -35,30 +36,6 @@
unsigned long huge_page_size;
unsigned long base_page_size;
-/*
- * default_huge_page_size copied from mlock2-tests.c
- */
-unsigned long default_huge_page_size(void)
-{
- unsigned long hps = 0;
- char *line = NULL;
- size_t linelen = 0;
- FILE *f = fopen("/proc/meminfo", "r");
-
- if (!f)
- return 0;
- while (getline(&line, &linelen, f) > 0) {
- if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
- hps <<= 10;
- break;
- }
- }
-
- free(line);
- fclose(f);
- return hps;
-}
-
unsigned long get_free_hugepages(void)
{
unsigned long fhp = 0;
diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
index 361ef7192cc6..380ab5f0a534 100644
--- a/tools/testing/selftests/mm/thuge-gen.c
+++ b/tools/testing/selftests/mm/thuge-gen.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include <stdarg.h>
#include <string.h>
+#include "vm_util.h"
#define err(x) perror(x), exit(1)
@@ -74,24 +75,6 @@ void find_pagesizes(void)
globfree(&g);
}
-unsigned long default_huge_page_size(void)
-{
- unsigned long hps = 0;
- char *line = NULL;
- size_t linelen = 0;
- FILE *f = fopen("/proc/meminfo", "r");
- if (!f)
- return 0;
- while (getline(&line, &linelen, f) > 0) {
- if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
- hps <<= 10;
- break;
- }
- }
- free(line);
- return hps;
-}
-
void show(unsigned long ps)
{
char buf[100];
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index a96d126cb40e..4cc80a0e8955 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -1703,30 +1703,6 @@ static int userfaultfd_stress(void)
|| userfaultfd_events_test() || userfaultfd_minor_test();
}
-/*
- * Copied from mlock2-tests.c
- */
-unsigned long default_huge_page_size(void)
-{
- unsigned long hps = 0;
- char *line = NULL;
- size_t linelen = 0;
- FILE *f = fopen("/proc/meminfo", "r");
-
- if (!f)
- return 0;
- while (getline(&line, &linelen, f) > 0) {
- if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
- hps <<= 10;
- break;
- }
- }
-
- free(line);
- fclose(f);
- return hps;
-}
-
static void set_test_type(const char *type)
{
if (!strcmp(type, "anon")) {
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 0204c469be43..12dc654b5be3 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -180,3 +180,24 @@ int64_t allocate_transhuge(void *ptr, int pagemap_fd)
return -1;
}
+
+unsigned long default_huge_page_size(void)
+{
+ unsigned long hps = 0;
+ char *line = NULL;
+ size_t linelen = 0;
+ FILE *f = fopen("/proc/meminfo", "r");
+
+ if (!f)
+ return 0;
+ while (getline(&line, &linelen, f) > 0) {
+ if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
+ hps <<= 10;
+ break;
+ }
+ }
+
+ free(line);
+ fclose(f);
+ return hps;
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 6edeb531afc6..d7163fff8fb7 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -35,6 +35,7 @@ bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);
bool check_huge_file(void *addr, int nr_hpages, uint64_t hpage_size);
bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
int64_t allocate_transhuge(void *ptr, int pagemap_fd);
+unsigned long default_huge_page_size(void);
/*
* On ppc64 this will only work with radix 2M hugepage size
--
2.39.1
We've got the macros in uffd-stress.c, move it over and use it in
vm_util.h.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/userfaultfd.c | 8 --------
tools/testing/selftests/mm/vm_util.c | 16 ++++------------
tools/testing/selftests/mm/vm_util.h | 8 ++++++++
3 files changed, 12 insertions(+), 20 deletions(-)
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index 4cc80a0e8955..7e841f7e2884 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -1389,14 +1389,6 @@ static int userfaultfd_minor_test(void)
return stats.missing_faults != 0 || stats.minor_faults != nr_pages;
}
-#define BIT_ULL(nr) (1ULL << (nr))
-#define PM_SOFT_DIRTY BIT_ULL(55)
-#define PM_MMAP_EXCLUSIVE BIT_ULL(56)
-#define PM_UFFD_WP BIT_ULL(57)
-#define PM_FILE BIT_ULL(61)
-#define PM_SWAP BIT_ULL(62)
-#define PM_PRESENT BIT_ULL(63)
-
static int pagemap_open(void)
{
int fd = open("/proc/self/pagemap", O_RDONLY);
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 12dc654b5be3..8e9da621764a 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -25,25 +25,17 @@ uint64_t pagemap_get_entry(int fd, char *start)
bool pagemap_is_softdirty(int fd, char *start)
{
- uint64_t entry = pagemap_get_entry(fd, start);
-
- // Check if dirty bit (55th bit) is set
- return entry & 0x0080000000000000ull;
+ return pagemap_get_entry(fd, start) & PM_SOFT_DIRTY;
}
bool pagemap_is_swapped(int fd, char *start)
{
- uint64_t entry = pagemap_get_entry(fd, start);
-
- return entry & 0x4000000000000000ull;
+ return pagemap_get_entry(fd, start) & PM_SWAP;
}
bool pagemap_is_populated(int fd, char *start)
{
- uint64_t entry = pagemap_get_entry(fd, start);
-
- /* Present or swapped. */
- return entry & 0xc000000000000000ull;
+ return pagemap_get_entry(fd, start) & (PM_PRESENT | PM_SWAP);
}
unsigned long pagemap_get_pfn(int fd, char *start)
@@ -51,7 +43,7 @@ unsigned long pagemap_get_pfn(int fd, char *start)
uint64_t entry = pagemap_get_entry(fd, start);
/* If present (63th bit), PFN is at bit 0 -- 54. */
- if (entry & 0x8000000000000000ull)
+ if (entry & PM_PRESENT)
return entry & 0x007fffffffffffffull;
return -1ul;
}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index d7163fff8fb7..d9fadddb5c69 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -6,6 +6,14 @@
#include <string.h> /* ffsl() */
#include <unistd.h> /* _SC_PAGESIZE */
+#define BIT_ULL(nr) (1ULL << (nr))
+#define PM_SOFT_DIRTY BIT_ULL(55)
+#define PM_MMAP_EXCLUSIVE BIT_ULL(56)
+#define PM_UFFD_WP BIT_ULL(57)
+#define PM_FILE BIT_ULL(61)
+#define PM_SWAP BIT_ULL(62)
+#define PM_PRESENT BIT_ULL(63)
+
extern unsigned int __page_size;
extern unsigned int __page_shift;
--
2.39.1
Dump a summary after running whatever test specified. Useful for human
runners to identify any kind of failures (besides exit code).
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/run_vmtests.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index c0f93b668c0c..9cc33984aa9f 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -5,6 +5,9 @@
# Kselftest framework requirement - SKIP code is 4.
ksft_skip=4
+count_pass=0
+count_fail=0
+count_skip=0
exitcode=0
usage() {
@@ -149,11 +152,14 @@ run_test() {
"$@"
local ret=$?
if [ $ret -eq 0 ]; then
+ count_pass=$(( $count_pass + 1 ))
echo "[PASS]"
elif [ $ret -eq $ksft_skip ]; then
+ count_skip=$(( $count_skip + 1 ))
echo "[SKIP]"
exitcode=$ksft_skip
else
+ count_fail=$(( $count_fail + 1 ))
echo "[FAIL]"
exitcode=1
fi
@@ -279,4 +285,6 @@ CATEGORY="soft_dirty" run_test ./soft-dirty
# COW tests
CATEGORY="cow" run_test ./cow
+echo "SUMMARY: PASS=${count_pass} SKIP=${count_skip} FAIL=${count_fail}"
+
exit $exitcode
--
2.39.1
We do have plenty of files that want to link against vm_util.c. Just make
it simple by linking it always.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/Makefile | 12 +-----------
1 file changed, 1 insertion(+), 11 deletions(-)
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 47516a78d447..b35f3eafde3c 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -105,17 +105,7 @@ TEST_FILES += va_128TBswitch.sh
include ../lib.mk
-$(OUTPUT)/cow: vm_util.c
-$(OUTPUT)/khugepaged: vm_util.c
-$(OUTPUT)/ksm_functional_tests: vm_util.c
-$(OUTPUT)/madv_populate: vm_util.c
-$(OUTPUT)/soft-dirty: vm_util.c
-$(OUTPUT)/split_huge_page_test: vm_util.c
-$(OUTPUT)/userfaultfd: vm_util.c
-$(OUTPUT)/gup_test: vm_util.c
-$(OUTPUT)/mrelease_test: vm_util.c
-$(OUTPUT)/transhuge-stress: vm_util.c
-$(OUTPUT)/ksm_tests: vm_util.c
+$(TEST_GEN_PROGS): vm_util.c
ifeq ($(MACHINE),x86_64)
BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
--
2.39.1
Make the check as simple as "test_type == TEST_HUGETLB" because that's the
only mem that doesn't support ZEROPAGE.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/userfaultfd.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index 795fbc4d84f8..d724f1c78847 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
{
struct uffdio_zeropage uffdio_zeropage;
int ret;
- bool has_zeropage = get_expected_ioctls(0) & (1 << _UFFDIO_ZEROPAGE);
+ bool has_zeropage = !(test_type == TEST_HUGETLB);
__s64 res;
if (offset >= nr_pages * page_size)
--
2.39.1
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/.gitignore | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
index 1f8c36a9fa10..347277f2adc3 100644
--- a/tools/testing/selftests/mm/.gitignore
+++ b/tools/testing/selftests/mm/.gitignore
@@ -36,3 +36,5 @@ split_huge_page_test
ksm_tests
local_config.h
local_config.mk
+ksm_functional_tests
+mdwe_test
--
2.39.1
Add two helpers to register/unregister to an uffd. Use them to drop
duplicate codes.
This patch also drops assert_expected_ioctls_present() and
get_expected_ioctls(). Reasons:
- It'll need a lot of effort to pass test_type==HUGETLB into it from the
upper, so it's the simplest way to get rid of another global var
- The ioctls returned in UFFDIO_REGISTER is hardly useful at all, because
any app can already detect kernel support on any ioctl via its
corresponding UFFD_FEATURE_*. The check here is for sanity mostly but
it's probably destined no user app will even use it.
- It's not friendly to one future goal of uffd to run on old kernels, the
problem is get_expected_ioctls() compiles against UFFD_API_RANGE_IOCTLS,
which is a value that can change depending on where the test is compiled,
rather than reflecting what the kernel underneath has. It means it'll
report false negatives on old kernels so it's against our will.
So let's make our live easier.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/hugepage-mremap.c | 7 +-
.../selftests/mm/ksm_functional_tests.c | 6 +-
tools/testing/selftests/mm/uffd-common.c | 28 +------
tools/testing/selftests/mm/uffd-common.h | 2 -
tools/testing/selftests/mm/uffd-stress.c | 83 +++++--------------
tools/testing/selftests/mm/vm_util.c | 37 +++++++++
tools/testing/selftests/mm/vm_util.h | 7 ++
7 files changed, 66 insertions(+), 104 deletions(-)
diff --git a/tools/testing/selftests/mm/hugepage-mremap.c b/tools/testing/selftests/mm/hugepage-mremap.c
index e53b5eaa8fce..2084692fe1c4 100644
--- a/tools/testing/selftests/mm/hugepage-mremap.c
+++ b/tools/testing/selftests/mm/hugepage-mremap.c
@@ -60,7 +60,6 @@ static void register_region_with_uffd(char *addr, size_t len)
{
long uffd; /* userfaultfd file descriptor */
struct uffdio_api uffdio_api;
- struct uffdio_register uffdio_register;
/* Create and enable userfaultfd object. */
@@ -96,11 +95,7 @@ static void register_region_with_uffd(char *addr, size_t len)
* handling by the userfaultfd object. In mode, we request to track
* missing pages (i.e., pages that have not yet been faulted in).
*/
-
- uffdio_register.range.start = (unsigned long)addr;
- uffdio_register.range.len = len;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1) {
+ if (uffd_register(uffd, addr, len, true, false, false)) {
perror("ioctl-UFFDIO_REGISTER");
exit(1);
}
diff --git a/tools/testing/selftests/mm/ksm_functional_tests.c b/tools/testing/selftests/mm/ksm_functional_tests.c
index d8b5b4930412..d3f26050dfd7 100644
--- a/tools/testing/selftests/mm/ksm_functional_tests.c
+++ b/tools/testing/selftests/mm/ksm_functional_tests.c
@@ -178,7 +178,6 @@ static void test_unmerge_discarded(void)
static void test_unmerge_uffd_wp(void)
{
struct uffdio_writeprotect uffd_writeprotect;
- struct uffdio_register uffdio_register;
const unsigned int size = 2 * MiB;
struct uffdio_api uffdio_api;
char *map;
@@ -210,10 +209,7 @@ static void test_unmerge_uffd_wp(void)
}
/* Register UFFD-WP, no need for an actual handler. */
- uffdio_register.range.start = (unsigned long) map;
- uffdio_register.range.len = size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) < 0) {
+ if (uffd_register(uffd, map, size, false, true, false)) {
ksft_test_result_fail("UFFDIO_REGISTER_MODE_WP failed\n");
goto close_uffd;
}
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index c57757c2a36f..17f2bb82c3db 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -330,33 +330,6 @@ void uffd_test_ctx_init(uint64_t features)
err("pipe");
}
-uint64_t get_expected_ioctls(uint64_t mode)
-{
- uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
-
- if (test_type == TEST_HUGETLB)
- ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
-
- if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
- ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
-
- if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
- ioctls &= ~(1 << _UFFDIO_CONTINUE);
-
- return ioctls;
-}
-
-void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
-{
- uint64_t expected = get_expected_ioctls(mode);
- uint64_t actual = ioctls & expected;
-
- if (actual != expected) {
- err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
- expected, actual);
- }
-}
-
void wp_range(int ufd, __u64 start, __u64 len, bool wp)
{
struct uffdio_writeprotect prms;
@@ -609,3 +582,4 @@ int copy_page(int ufd, unsigned long offset)
{
return __copy_page(ufd, offset, false);
}
+
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index d9430cfdcb19..11f770391bd9 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -101,8 +101,6 @@ extern uffd_test_ops_t *uffd_test_ops;
void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
void uffd_test_ctx_init(uint64_t features);
void userfaultfd_open(uint64_t *features);
-uint64_t get_expected_ioctls(uint64_t mode);
-void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index c68a9aeefc41..e6d39a755082 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -463,28 +463,19 @@ static int uffdio_zeropage(int ufd, unsigned long offset)
/* exercise UFFDIO_ZEROPAGE */
static int userfaultfd_zeropage_test(void)
{
- struct uffdio_register uffdio_register;
-
printf("testing UFFDIO_ZEROPAGE: ");
fflush(stdout);
uffd_test_ctx_init(0);
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, test_uffdio_wp, false))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
-
if (area_dst_alias) {
/* Needed this to test zeropage-retry on shared memory */
- uffdio_register.range.start = (unsigned long) area_dst_alias;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
+ true, test_uffdio_wp, false))
err("register failure");
}
@@ -498,7 +489,6 @@ static int userfaultfd_zeropage_test(void)
static int userfaultfd_events_test(void)
{
- struct uffdio_register uffdio_register;
pthread_t uffd_mon;
int err, features;
pid_t pid;
@@ -514,17 +504,10 @@ static int userfaultfd_events_test(void)
fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, test_uffdio_wp, false))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
-
if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
err("uffd_poll_thread create");
@@ -550,7 +533,6 @@ static int userfaultfd_events_test(void)
static int userfaultfd_sig_test(void)
{
- struct uffdio_register uffdio_register;
unsigned long userfaults;
pthread_t uffd_mon;
int err, features;
@@ -566,17 +548,10 @@ static int userfaultfd_sig_test(void)
fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, test_uffdio_wp, false))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
-
if (faulting_process(1))
err("faulting process failed");
@@ -629,7 +604,6 @@ void check_memory_contents(char *p)
static int userfaultfd_minor_test(void)
{
unsigned long p;
- struct uffdio_register uffdio_register;
pthread_t uffd_mon;
char c;
struct uffd_stats stats = { 0 };
@@ -642,17 +616,10 @@ static int userfaultfd_minor_test(void)
uffd_test_ctx_init(uffd_minor_feature());
- uffdio_register.range.start = (unsigned long)area_dst_alias;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MINOR;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
+ false, test_uffdio_wp, true))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
-
/*
* After registering with UFFD, populate the non-UFFD-registered side of
* the shared mapping. This should *not* trigger any UFFD minor faults.
@@ -777,7 +744,6 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
static void userfaultfd_pagemap_test(unsigned int test_pgsize)
{
- struct uffdio_register uffdio_register;
int pagemap_fd;
uint64_t value;
@@ -805,10 +771,8 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
err("madvise(MADV_NOHUGEPAGE) failed");
}
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ false, true, false))
err("register failed");
pagemap_fd = pagemap_open();
@@ -858,8 +822,8 @@ static int userfaultfd_stress(void)
{
void *area;
unsigned long nr;
- struct uffdio_register uffdio_register;
struct uffd_stats uffd_stats[nr_cpus];
+ uint64_t mem_size = nr_pages * page_size;
uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
@@ -894,20 +858,13 @@ static int userfaultfd_stress(void)
fcntl(uffd, F_SETFL, uffd_flags & ~O_NONBLOCK);
/* register */
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, mem_size,
+ true, test_uffdio_wp, false))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
if (area_dst_alias) {
- uffdio_register.range.start = (unsigned long)
- area_dst_alias;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst_alias, mem_size,
+ true, test_uffdio_wp, false))
err("register failure alias");
}
@@ -949,12 +906,10 @@ static int userfaultfd_stress(void)
nr_pages * page_size, false);
/* unregister */
- if (ioctl(uffd, UFFDIO_UNREGISTER, &uffdio_register.range))
+ if (uffd_unregister(uffd, area_dst, mem_size))
err("unregister failure");
if (area_dst_alias) {
- uffdio_register.range.start = (unsigned long) area_dst;
- if (ioctl(uffd, UFFDIO_UNREGISTER,
- &uffdio_register.range))
+ if (uffd_unregister(uffd, area_dst_alias, mem_size))
err("unregister failure alias");
}
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 8e9da621764a..10e76400ed70 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
#include <string.h>
#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <linux/userfaultfd.h>
#include "../kselftest.h"
#include "vm_util.h"
@@ -193,3 +195,38 @@ unsigned long default_huge_page_size(void)
fclose(f);
return hps;
}
+
+int uffd_register(int uffd, void *addr, uint64_t len,
+ bool miss, bool wp, bool minor)
+{
+ struct uffdio_register uffdio_register = { 0 };
+ uint64_t mode = 0;
+ int ret = 0;
+
+ if (miss)
+ mode |= UFFDIO_REGISTER_MODE_MISSING;
+ if (wp)
+ mode |= UFFDIO_REGISTER_MODE_WP;
+ if (minor)
+ mode |= UFFDIO_REGISTER_MODE_MINOR;
+
+ uffdio_register.range.start = (unsigned long)addr;
+ uffdio_register.range.len = len;
+ uffdio_register.mode = mode;
+
+ if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1)
+ ret = -errno;
+
+ return ret;
+}
+
+int uffd_unregister(int uffd, void *addr, uint64_t len)
+{
+ struct uffdio_range range = { .start = (uintptr_t)addr, .len = len };
+ int ret = 0;
+
+ if (ioctl(uffd, UFFDIO_UNREGISTER, &range) == -1)
+ ret = -errno;
+
+ return ret;
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index d9fadddb5c69..a67db8432855 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -45,6 +45,13 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
int64_t allocate_transhuge(void *ptr, int pagemap_fd);
unsigned long default_huge_page_size(void);
+int uffd_open_dev(unsigned int flags);
+int uffd_open_sys(unsigned int flags);
+int uffd_open(unsigned int flags);
+int uffd_register(int uffd, void *addr, uint64_t len,
+ bool miss, bool wp, bool minor);
+int uffd_unregister(int uffd, void *addr, uint64_t len);
+
/*
* On ppc64 this will only work with radix 2M hugepage size
*/
--
2.39.1
In many ways it's weird and unwanted to keep all the tests in the same
userfaultfd.c at least when still in the current way.
For example, it doesn't make much sense to run the stress test for each
method we can create an userfaultfd handle (either via syscall or /dev/
node). It's a waste of time running this twice for the whole stress as the
stress paths are the same, only the open path is different.
It's also just weird to need to manually specify different types of memory
to run all unit tests for the userfaultfd interface. We should be able to
just run a single program and that should go through all functional uffd
tests without running the stress test at all. The stress test was more for
torturing and finding race conditions. We don't want to wait for stress to
finish just to regress test a functional test.
When we start to pile up more things on top of the same file and same
functions, things start to go a bit chaos and the code is just harder to
maintain too with tons of global variables.
This patch creates a new test uffd-unit-tests to keep userfaultfd unit
tests in the future, currently empty.
Meanwhile rename the old userfaultfd.c test to uffd-stress.c.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/.gitignore | 3 ++-
tools/testing/selftests/mm/Makefile | 8 +++---
tools/testing/selftests/mm/run_vmtests.sh | 10 ++++---
.../mm/{userfaultfd.c => uffd-stress.c} | 0
tools/testing/selftests/mm/uffd-unit-tests.c | 27 +++++++++++++++++++
5 files changed, 40 insertions(+), 8 deletions(-)
rename tools/testing/selftests/mm/{userfaultfd.c => uffd-stress.c} (100%)
create mode 100644 tools/testing/selftests/mm/uffd-unit-tests.c
diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
index 347277f2adc3..8917455f4f51 100644
--- a/tools/testing/selftests/mm/.gitignore
+++ b/tools/testing/selftests/mm/.gitignore
@@ -21,7 +21,8 @@ protection_keys
protection_keys_32
protection_keys_64
madv_populate
-userfaultfd
+uffd-stress
+uffd-unit-tests
mlock-intersect-test
mlock-random-test
virtual_address_range
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 9c3737285f8a..ca8bc4ac71e6 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -20,7 +20,7 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p
# Avoid accidental wrong builds, due to built-in rules working just a little
# bit too well--but not quite as well as required for our situation here.
#
-# In other words, "make userfaultfd" is supposed to fail to build at all,
+# In other words, "make $SOME_TEST" is supposed to fail to build at all,
# because this Makefile only supports either "make" (all), or "make /full/path".
# However, the built-in rules, if not suppressed, will pick up CFLAGS and the
# initial LDLIBS (but not the target-specific LDLIBS, because those are only
@@ -56,7 +56,8 @@ TEST_GEN_PROGS += mremap_test
TEST_GEN_PROGS += on-fault-limit
TEST_GEN_PROGS += thuge-gen
TEST_GEN_PROGS += transhuge-stress
-TEST_GEN_PROGS += userfaultfd
+TEST_GEN_PROGS += uffd-stress
+TEST_GEN_PROGS += uffd-unit-tests
TEST_GEN_PROGS += soft-dirty
TEST_GEN_PROGS += split_huge_page_test
TEST_GEN_PROGS += ksm_tests
@@ -107,7 +108,8 @@ include ../lib.mk
$(TEST_GEN_PROGS): vm_util.c
-$(OUTPUT)/userfaultfd: uffd-common.c
+$(OUTPUT)/uffd-stress: uffd-common.c
+$(OUTPUT)/uffd-unit-tests: uffd-common.c
ifeq ($(MACHINE),x86_64)
BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index 9cc33984aa9f..af7bbc74cd83 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -196,14 +196,16 @@ CATEGORY="gup_test" run_test ./gup_test -a
# Dump pages 0, 19, and 4096, using pin_user_pages:
CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000
+CATEGORY="userfaultfd" run_test ./uffd-unit-tests
uffd_mods=("" ":dev")
+uffd_stress_bin=./uffd-stress
for mod in "${uffd_mods[@]}"; do
- CATEGORY="userfaultfd" run_test ./userfaultfd anon${mod} 20 16
+ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16
# Hugetlb tests require source and destination huge pages. Pass in half
# the size ($half_ufd_size_MB), which is used for *each*.
- CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32
- CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32
- CATEGORY="userfaultfd" run_test ./userfaultfd shmem${mod} 20 16
+ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32
+ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32
+ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16
done
#cleanup
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/uffd-stress.c
similarity index 100%
rename from tools/testing/selftests/mm/userfaultfd.c
rename to tools/testing/selftests/mm/uffd-stress.c
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
new file mode 100644
index 000000000000..6857388783be
--- /dev/null
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Userfaultfd unit tests.
+ *
+ * Copyright (C) 2015-2023 Red Hat, Inc.
+ */
+
+#include "uffd-common.h"
+
+#ifdef __NR_userfaultfd
+
+int main(int argc, char *argv[])
+{
+ return KSFT_PASS;
+}
+
+#else /* __NR_userfaultfd */
+
+#warning "missing __NR_userfaultfd definition"
+
+int main(void)
+{
+ printf("Skipping %s (missing __NR_userfaultfd)\n", __file__);
+ return KSFT_SKIP;
+}
+
+#endif /* __NR_userfaultfd */
--
2.39.1
TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the mm/Makefile to
specify programs that need to build. Logically all these binaries should
all fall into TEST_GEN_PROGS.
Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can reference
all the tests easily later.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/Makefile | 63 +++++++++++++++--------------
1 file changed, 32 insertions(+), 31 deletions(-)
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 4188435967ed..47516a78d447 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -31,34 +31,35 @@ MAKEFLAGS += --no-builtin-rules
CFLAGS = -Wall -I $(top_srcdir) -I $(top_srcdir)/tools/include/uapi $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
LDLIBS = -lrt -lpthread
-TEST_GEN_FILES = cow
-TEST_GEN_FILES += compaction_test
-TEST_GEN_FILES += gup_test
-TEST_GEN_FILES += hmm-tests
-TEST_GEN_FILES += hugetlb-madvise
-TEST_GEN_FILES += hugepage-mmap
-TEST_GEN_FILES += hugepage-mremap
-TEST_GEN_FILES += hugepage-shm
-TEST_GEN_FILES += hugepage-vmemmap
-TEST_GEN_FILES += khugepaged
+
+TEST_GEN_PROGS = cow
+TEST_GEN_PROGS += compaction_test
+TEST_GEN_PROGS += gup_test
+TEST_GEN_PROGS += hmm-tests
+TEST_GEN_PROGS += hugetlb-madvise
+TEST_GEN_PROGS += hugepage-mmap
+TEST_GEN_PROGS += hugepage-mremap
+TEST_GEN_PROGS += hugepage-shm
+TEST_GEN_PROGS += hugepage-vmemmap
+TEST_GEN_PROGS += khugepaged
TEST_GEN_PROGS = madv_populate
-TEST_GEN_FILES += map_fixed_noreplace
-TEST_GEN_FILES += map_hugetlb
-TEST_GEN_FILES += map_populate
-TEST_GEN_FILES += memfd_secret
-TEST_GEN_FILES += migration
-TEST_GEN_FILES += mlock-random-test
-TEST_GEN_FILES += mlock2-tests
-TEST_GEN_FILES += mrelease_test
-TEST_GEN_FILES += mremap_dontunmap
-TEST_GEN_FILES += mremap_test
-TEST_GEN_FILES += on-fault-limit
-TEST_GEN_FILES += thuge-gen
-TEST_GEN_FILES += transhuge-stress
-TEST_GEN_FILES += userfaultfd
+TEST_GEN_PROGS += map_fixed_noreplace
+TEST_GEN_PROGS += map_hugetlb
+TEST_GEN_PROGS += map_populate
+TEST_GEN_PROGS += memfd_secret
+TEST_GEN_PROGS += migration
+TEST_GEN_PROGS += mlock-random-test
+TEST_GEN_PROGS += mlock2-tests
+TEST_GEN_PROGS += mrelease_test
+TEST_GEN_PROGS += mremap_dontunmap
+TEST_GEN_PROGS += mremap_test
+TEST_GEN_PROGS += on-fault-limit
+TEST_GEN_PROGS += thuge-gen
+TEST_GEN_PROGS += transhuge-stress
+TEST_GEN_PROGS += userfaultfd
TEST_GEN_PROGS += soft-dirty
TEST_GEN_PROGS += split_huge_page_test
-TEST_GEN_FILES += ksm_tests
+TEST_GEN_PROGS += ksm_tests
TEST_GEN_PROGS += ksm_functional_tests
TEST_GEN_PROGS += mdwe_test
@@ -76,24 +77,24 @@ CFLAGS += -no-pie
endif
ifeq ($(CAN_BUILD_I386),1)
-TEST_GEN_FILES += $(BINARIES_32)
+TEST_GEN_PROGS += $(BINARIES_32)
endif
ifeq ($(CAN_BUILD_X86_64),1)
-TEST_GEN_FILES += $(BINARIES_64)
+TEST_GEN_PROGS += $(BINARIES_64)
endif
else
ifneq (,$(findstring $(MACHINE),ppc64))
-TEST_GEN_FILES += protection_keys
+TEST_GEN_PROGS += protection_keys
endif
endif
ifneq (,$(filter $(MACHINE),arm64 ia64 mips64 parisc64 ppc64 riscv64 s390x sh64 sparc64 x86_64))
-TEST_GEN_FILES += va_128TBswitch
-TEST_GEN_FILES += virtual_address_range
-TEST_GEN_FILES += write_to_hugetlbfs
+TEST_GEN_PROGS += va_128TBswitch
+TEST_GEN_PROGS += virtual_address_range
+TEST_GEN_PROGS += write_to_hugetlbfs
endif
TEST_PROGS := run_vmtests.sh
--
2.39.1
hpage_size was wrongly used. Sometimes it means hugetlb default size,
sometimes it was used as thp size.
Remove the global variable and use the right one at each place.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-common.c | 7 ++++---
tools/testing/selftests/mm/uffd-common.h | 2 +-
tools/testing/selftests/mm/uffd-stress.c | 6 +++---
3 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index a2b6e4957d0f..025e40ffc7bf 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -10,7 +10,7 @@
#define BASE_PMD_ADDR ((void *)(1UL << 30))
volatile bool test_uffdio_copy_eexist = true;
-unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
+unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
int uffd = -1, uffd_flags, finished, *pipefd, test_type;
bool map_shared, test_collapse, test_dev_userfaultfd;
@@ -115,7 +115,7 @@ static void shmem_release_pages(char *rel_area)
static void shmem_allocate_area(void **alloc_area, bool is_src)
{
void *area_alias = NULL;
- size_t bytes = nr_pages * page_size;
+ size_t bytes = nr_pages * page_size, hpage_size = read_pmd_pagesize();
unsigned long offset = is_src ? 0 : bytes;
char *p = NULL, *p_alias = NULL;
int mem_fd = uffd_mem_fd_create(bytes * 2, false);
@@ -159,7 +159,8 @@ static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
{
- if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
+ if (!check_huge_shmem(area_dst_alias, expect_nr_hpages,
+ read_pmd_pagesize()))
err("Did not find expected %d number of hugepages",
expect_nr_hpages);
}
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 0dfab7057295..47565b2f2dee 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -85,7 +85,7 @@ struct uffd_test_ops {
};
typedef struct uffd_test_ops uffd_test_ops_t;
-extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
+extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
extern int uffd, uffd_flags, finished, *pipefd, test_type;
extern bool map_shared, test_collapse, test_dev_userfaultfd;
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 4eca1a0276c2..54fc9b4ffa3c 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -655,7 +655,7 @@ static int userfaultfd_minor_test(void)
uffd_test_ops->check_pmd_mapping(area_dst,
nr_pages * page_size /
- hpage_size);
+ read_pmd_pagesize());
/*
* This won't cause uffd-fault - it purely just makes sure there
* was no corruption.
@@ -997,7 +997,7 @@ static void parse_test_type_arg(const char *raw_type)
err("Unsupported test: %s", raw_type);
if (test_type == TEST_HUGETLB)
- page_size = hpage_size;
+ page_size = default_huge_page_size();
else
page_size = sysconf(_SC_PAGE_SIZE);
@@ -1035,6 +1035,7 @@ static void sigalrm(int sig)
int main(int argc, char **argv)
{
size_t bytes;
+ size_t hpage_size = read_pmd_pagesize();
if (argc < 4)
usage();
@@ -1043,7 +1044,6 @@ int main(int argc, char **argv)
err("failed to arm SIGALRM");
alarm(ALARM_INTERVAL_SECS);
- hpage_size = default_huge_page_size();
parse_test_type_arg(argv[1]);
bytes = atol(argv[2]) * 1024 * 1024;
--
2.39.1
Add one simple test for UFFDIO_API. With that, I also added a bunch of
small but handy helpers along the way.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 111 ++++++++++++++++++-
tools/testing/selftests/mm/vm_util.c | 10 ++
2 files changed, 120 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 6857388783be..dfb44ffad5f5 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -9,9 +9,118 @@
#ifdef __NR_userfaultfd
+struct {
+ unsigned int pass, skip, fail, total;
+} uffd_test_acct;
+
+static void uffd_test_report(void)
+{
+ printf("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n",
+ uffd_test_acct.pass,
+ uffd_test_acct.skip,
+ uffd_test_acct.fail,
+ uffd_test_acct.total);
+}
+
+static void uffd_test_pass(void)
+{
+ printf("done\n");
+ uffd_test_acct.pass++;
+}
+
+#define uffd_test_start(...) do { \
+ printf(__VA_ARGS__); \
+ printf("... "); \
+ uffd_test_acct.total++; \
+ } while (0)
+
+#define uffd_test_fail(...) do { \
+ printf("failed [reason: "); \
+ printf(__VA_ARGS__); \
+ printf("]\n"); \
+ uffd_test_acct.fail++; \
+ } while (0)
+
+#define uffd_test_skip(...) do { \
+ printf("skipped [reason: "); \
+ printf(__VA_ARGS__); \
+ printf("]\n"); \
+ uffd_test_acct.skip++; \
+ } while (0)
+
+/*
+ * Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
+ * return 1 even if some test failed as long as uffd supported, because in
+ * that case we still want to proceed with the rest uffd unit tests.
+ */
+static int test_uffd_api(bool use_dev)
+{
+ struct uffdio_api uffdio_api;
+ int uffd;
+
+ uffd_test_start("UFFDIO_API (with %s)",
+ use_dev ? "/dev/userfaultfd" : "syscall");
+
+ if (use_dev)
+ uffd = uffd_open_dev(UFFD_FLAGS);
+ else
+ uffd = uffd_open_sys(UFFD_FLAGS);
+ if (uffd < 0) {
+ uffd_test_skip("cannot open userfaultfd handle");
+ return 0;
+ }
+
+ /* Test wrong UFFD_API */
+ uffdio_api.api = 0xab;
+ uffdio_api.features = 0;
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
+ uffd_test_fail("UFFDIO_API should fail with wrong api but didn't");
+ goto out;
+ }
+
+ /* Test wrong feature bit */
+ uffdio_api.api = UFFD_API;
+ uffdio_api.features = BIT_ULL(63);
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
+ uffd_test_fail("UFFDIO_API should fail with wrong feature but didn't");
+ goto out;
+ }
+
+ /* Test normal UFFDIO_API */
+ uffdio_api.api = UFFD_API;
+ uffdio_api.features = 0;
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api)) {
+ uffd_test_fail("UFFDIO_API should succeed but failed");
+ goto out;
+ }
+
+ /* Test double requests of UFFDIO_API with a random feature set */
+ uffdio_api.features = BIT_ULL(0);
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
+ uffd_test_fail("UFFDIO_API should reject initialized uffd");
+ goto out;
+ }
+
+ uffd_test_pass();
+out:
+ close(uffd);
+ /* We have a valid uffd handle */
+ return 1;
+}
+
int main(int argc, char *argv[])
{
- return KSFT_PASS;
+ int has_uffd;
+
+ has_uffd = test_uffd_api(false);
+ has_uffd |= test_uffd_api(true);
+
+ if (!has_uffd) {
+ printf("Userfaultfd not supported or unprivileged, skip all tests\n");
+ exit(KSFT_SKIP);
+ }
+ uffd_test_report();
+ return uffd_test_acct.fail ? KSFT_FAIL : KSFT_PASS;
}
#else /* __NR_userfaultfd */
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 7c2bf88d6393..62fcf039d6b7 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -254,3 +254,13 @@ int uffd_open_sys(unsigned int flags)
return -1;
#endif
}
+
+int uffd_open(unsigned int flags)
+{
+ int uffd = uffd_open_sys(flags);
+
+ if (uffd < 0)
+ uffd = uffd_open_dev(flags);
+
+ return uffd;
+}
--
2.39.1
Drop it by creating the memfd dynamically in the tests.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-common.c | 28 +++++++++++++++++++++++-
tools/testing/selftests/mm/uffd-common.h | 2 +-
tools/testing/selftests/mm/uffd-stress.c | 15 -------------
3 files changed, 28 insertions(+), 17 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 3a9b5c1aca9d..a2b6e4957d0f 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -12,12 +12,32 @@
volatile bool test_uffdio_copy_eexist = true;
unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
-int mem_fd, uffd = -1, uffd_flags, finished, *pipefd, test_type;
+int uffd = -1, uffd_flags, finished, *pipefd, test_type;
bool map_shared, test_collapse, test_dev_userfaultfd;
bool test_uffdio_wp = true, test_uffdio_minor = false;
unsigned long long *count_verify;
uffd_test_ops_t *uffd_test_ops;
+static int uffd_mem_fd_create(off_t mem_size, bool hugetlb)
+{
+ unsigned int memfd_flags = 0;
+ int mem_fd;
+
+ if (hugetlb)
+ memfd_flags = MFD_HUGETLB;
+ mem_fd = memfd_create("uffd-test", memfd_flags);
+ if (mem_fd < 0)
+ err("memfd_create");
+ if (ftruncate(mem_fd, mem_size))
+ err("ftruncate");
+ if (fallocate(mem_fd,
+ FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0,
+ mem_size))
+ err("fallocate");
+
+ return mem_fd;
+}
+
static void anon_release_pages(char *rel_area)
{
if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
@@ -51,6 +71,7 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
off_t offset = is_src ? 0 : size;
void *area_alias = NULL;
char **alloc_area_alias;
+ int mem_fd = uffd_mem_fd_create(size * 2, true);
*alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
(map_shared ? MAP_SHARED : MAP_PRIVATE) |
@@ -73,6 +94,8 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
}
if (area_alias)
*alloc_area_alias = area_alias;
+
+ close(mem_fd);
}
static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
@@ -95,6 +118,7 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
size_t bytes = nr_pages * page_size;
unsigned long offset = is_src ? 0 : bytes;
char *p = NULL, *p_alias = NULL;
+ int mem_fd = uffd_mem_fd_create(bytes * 2, false);
if (test_collapse) {
p = BASE_PMD_ADDR;
@@ -124,6 +148,8 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
area_src_alias = area_alias;
else
area_dst_alias = area_alias;
+
+ close(mem_fd);
}
static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 11f770391bd9..0dfab7057295 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -87,7 +87,7 @@ typedef struct uffd_test_ops uffd_test_ops_t;
extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
-extern int mem_fd, uffd, uffd_flags, finished, *pipefd, test_type;
+extern int uffd, uffd_flags, finished, *pipefd, test_type;
extern bool map_shared, test_collapse, test_dev_userfaultfd;
extern bool test_uffdio_wp, test_uffdio_minor;
extern unsigned long long *count_verify;
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index e6d39a755082..4eca1a0276c2 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -1090,21 +1090,6 @@ int main(int argc, char **argv)
}
nr_pages = nr_pages_per_cpu * nr_cpus;
- if (test_type == TEST_SHMEM || test_type == TEST_HUGETLB) {
- unsigned int memfd_flags = 0;
-
- if (test_type == TEST_HUGETLB)
- memfd_flags = MFD_HUGETLB;
- mem_fd = memfd_create(argv[0], memfd_flags);
- if (mem_fd < 0)
- err("memfd_create");
- if (ftruncate(mem_fd, nr_pages * page_size * 2))
- err("ftruncate");
- if (fallocate(mem_fd,
- FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0,
- nr_pages * page_size * 2))
- err("fallocate");
- }
printf("nr_pages: %lu, nr_pages_per_cpu: %lu\n",
nr_pages, nr_pages_per_cpu);
return userfaultfd_stress();
--
2.39.1
Meanwhile drop pagemap_read_vaddr().
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/userfaultfd.c | 31 +++++++-----------------
1 file changed, 9 insertions(+), 22 deletions(-)
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index 7e841f7e2884..795fbc4d84f8 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -1399,19 +1399,6 @@ static int pagemap_open(void)
return fd;
}
-static uint64_t pagemap_read_vaddr(int fd, void *vaddr)
-{
- uint64_t value;
- int ret;
-
- ret = pread(fd, &value, sizeof(uint64_t),
- ((uint64_t)vaddr >> 12) * sizeof(uint64_t));
- if (ret != sizeof(uint64_t))
- err("pread() on pagemap failed");
-
- return value;
-}
-
/* This macro let __LINE__ works in err() */
#define pagemap_check_wp(value, wp) do { \
if (!!(value & PM_UFFD_WP) != wp) \
@@ -1427,7 +1414,7 @@ static int pagemap_test_fork(bool present)
if (!child) {
/* Open the pagemap fd of the child itself */
fd = pagemap_open();
- value = pagemap_read_vaddr(fd, area_dst);
+ value = pagemap_get_entry(fd, area_dst);
/*
* After fork() uffd-wp bit should be gone as long as we're
* without UFFD_FEATURE_EVENT_FORK
@@ -1446,24 +1433,24 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
/* Test applying pte marker to anon unpopulated */
wp_range(uffd, (uint64_t)area_dst, page_size, true);
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, true);
/* Test unprotect on anon pte marker */
wp_range(uffd, (uint64_t)area_dst, page_size, false);
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
/* Test zap on anon marker */
wp_range(uffd, (uint64_t)area_dst, page_size, true);
if (madvise(area_dst, page_size, MADV_DONTNEED))
err("madvise(MADV_DONTNEED) failed");
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
/* Test fault in after marker removed */
*area_dst = 1;
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
/* Drop it to make pte none again */
if (madvise(area_dst, page_size, MADV_DONTNEED))
@@ -1522,7 +1509,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
/* Touch the page */
*area_dst = 1;
wp_range(uffd, (uint64_t)area_dst, test_pgsize, true);
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, true);
/* Make sure uffd-wp bit dropped when fork */
if (pagemap_test_fork(true))
@@ -1536,7 +1523,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
err("madvise(MADV_PAGEOUT) failed");
/* Uffd-wp should persist even swapped out */
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, true);
/* Make sure uffd-wp bit dropped when fork */
if (pagemap_test_fork(false))
@@ -1544,12 +1531,12 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
/* Unprotect; this tests swap pte modifications */
wp_range(uffd, (uint64_t)area_dst, page_size, false);
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
/* Fault in the page from disk */
*area_dst = 2;
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
close(pagemap_fd);
--
2.39.1
The idea was trying to flip this var in the alarm handler from time to time
to test -EEXIST of UFFDIO_ZEROPAGE, but firstly it's only used in the
zeropage test so probably only used once, meanwhile we passed
"retry==false" so it'll never got tested anyway.
Drop both sides so we always test UFFDIO_ZEROPAGE retries if has_zeropage
is set (!hugetlb).
One more thing to do is doing UFFDIO_REGISTER for the alias buffer too,
because otherwise the test won't even pass! We were just lucky that this
test never really got ran at all.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/userfaultfd.c | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index d724f1c78847..3487ec0bfcc8 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -88,7 +88,6 @@ static bool test_dev_userfaultfd;
/* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */
#define ALARM_INTERVAL_SECS 10
static volatile bool test_uffdio_copy_eexist = true;
-static volatile bool test_uffdio_zeropage_eexist = true;
/* Whether to test uffd write-protection */
static bool test_uffdio_wp = true;
/* Whether to test uffd minor faults */
@@ -1114,7 +1113,7 @@ static void retry_uffdio_zeropage(int ufd,
}
}
-static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
+static int __uffdio_zeropage(int ufd, unsigned long offset)
{
struct uffdio_zeropage uffdio_zeropage;
int ret;
@@ -1138,11 +1137,8 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
if (res != page_size) {
err("UFFDIO_ZEROPAGE unexpected size");
} else {
- if (test_uffdio_zeropage_eexist && retry) {
- test_uffdio_zeropage_eexist = false;
- retry_uffdio_zeropage(ufd, &uffdio_zeropage,
- offset);
- }
+ retry_uffdio_zeropage(ufd, &uffdio_zeropage,
+ offset);
return 1;
}
} else
@@ -1153,7 +1149,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
static int uffdio_zeropage(int ufd, unsigned long offset)
{
- return __uffdio_zeropage(ufd, offset, false);
+ return __uffdio_zeropage(ufd, offset);
}
/* exercise UFFDIO_ZEROPAGE */
@@ -1177,6 +1173,13 @@ static int userfaultfd_zeropage_test(void)
assert_expected_ioctls_present(
uffdio_register.mode, uffdio_register.ioctls);
+ if (area_dst_alias) {
+ /* Needed this to test zeropage-retry on shared memory */
+ uffdio_register.range.start = (unsigned long) area_dst_alias;
+ if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ err("register failure");
+ }
+
if (uffdio_zeropage(uffd, 0))
if (my_bcmp(area_dst, zeropage, page_size))
err("zeropage is not zero");
@@ -1763,7 +1766,6 @@ static void sigalrm(int sig)
if (sig != SIGALRM)
abort();
test_uffdio_copy_eexist = true;
- test_uffdio_zeropage_eexist = true;
alarm(ALARM_INTERVAL_SECS);
}
--
2.39.1
Provide two helpers to open an uffd handle. Drop the error checks around
SKIPs because it's inside an errexit() anyway, which IMHO doesn't really
help much if the test will not continue.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-common.c | 28 +++++-------------------
tools/testing/selftests/mm/vm_util.c | 24 ++++++++++++++++++++
2 files changed, 29 insertions(+), 23 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 17f2bb82c3db..3a9b5c1aca9d 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -192,34 +192,16 @@ void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
printf("\n");
}
-static int __userfaultfd_open_dev(void)
-{
- int fd, _uffd;
-
- fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
- if (fd < 0)
- errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
-
- _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
- if (_uffd < 0)
- errexit(errno == ENOTTY ? KSFT_SKIP : 1,
- "creating userfaultfd failed");
- close(fd);
- return _uffd;
-}
-
void userfaultfd_open(uint64_t *features)
{
struct uffdio_api uffdio_api;
if (test_dev_userfaultfd)
- uffd = __userfaultfd_open_dev();
- else {
- uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
- if (uffd < 0)
- errexit(errno == ENOSYS ? KSFT_SKIP : 1,
- "creating userfaultfd failed");
- }
+ uffd = uffd_open_dev(UFFD_FLAGS);
+ else
+ uffd = uffd_open_sys(UFFD_FLAGS);
+ if (uffd < 0)
+ err("uffd open failed (dev=%d)", test_dev_userfaultfd);
uffd_flags = fcntl(uffd, F_GETFD, NULL);
uffdio_api.api = UFFD_API;
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 10e76400ed70..7c2bf88d6393 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -3,6 +3,8 @@
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/userfaultfd.h>
+#include <sys/syscall.h>
+#include <unistd.h>
#include "../kselftest.h"
#include "vm_util.h"
@@ -230,3 +232,25 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
return ret;
}
+
+int uffd_open_dev(unsigned int flags)
+{
+ int fd, uffd;
+
+ fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+ if (fd < 0)
+ return fd;
+ uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
+ close(fd);
+
+ return uffd;
+}
+
+int uffd_open_sys(unsigned int flags)
+{
+#ifdef __NR_userfaultfd
+ return syscall(__NR_userfaultfd, flags);
+#else
+ return -1;
+#endif
+}
--
2.39.1
Add a framework to be prepared to move unit tests from uffd-stress.c into
uffd-unit-tests.c. The goal is to allow detection of uffd features for
each test, and also loop over specified types of memory that a test support.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 125 +++++++++++++++++++
tools/testing/selftests/mm/vm_util.c | 27 ++++
tools/testing/selftests/mm/vm_util.h | 1 +
3 files changed, 153 insertions(+)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index dfb44ffad5f5..007145063363 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -9,6 +9,66 @@
#ifdef __NR_userfaultfd
+/* The unit test doesn't need a large or random size, make it 32MB for now */
+#define UFFD_TEST_MEM_SIZE (32UL << 20)
+
+#define MEM_ANON BIT_ULL(0)
+#define MEM_SHMEM BIT_ULL(1)
+#define MEM_SHMEM_PRIVATE BIT_ULL(2)
+#define MEM_HUGETLB BIT_ULL(3)
+#define MEM_HUGETLB_PRIVATE BIT_ULL(4)
+
+struct mem_type {
+ const char *name;
+ unsigned int mem_flag;
+ uffd_test_ops_t *mem_ops;
+ bool shared;
+};
+typedef struct mem_type mem_type_t;
+
+mem_type_t mem_types[] = {
+ {
+ .name = "anon",
+ .mem_flag = MEM_ANON,
+ .mem_ops = &anon_uffd_test_ops,
+ .shared = false,
+ },
+ {
+ .name = "shmem",
+ .mem_flag = MEM_SHMEM,
+ .mem_ops = &shmem_uffd_test_ops,
+ .shared = true,
+ },
+ {
+ .name = "shmem-private",
+ .mem_flag = MEM_SHMEM_PRIVATE,
+ .mem_ops = &shmem_uffd_test_ops,
+ .shared = false,
+ },
+ {
+ .name = "hugetlb",
+ .mem_flag = MEM_HUGETLB,
+ .mem_ops = &hugetlb_uffd_test_ops,
+ .shared = true,
+ },
+ {
+ .name = "hugetlb-private",
+ .mem_flag = MEM_HUGETLB_PRIVATE,
+ .mem_ops = &hugetlb_uffd_test_ops,
+ .shared = false,
+ },
+};
+
+/* Returns: UFFD_TEST_* */
+typedef void (*uffd_test_fn)(void);
+
+typedef struct {
+ const char *name;
+ uffd_test_fn uffd_fn;
+ unsigned int mem_targets;
+ uint64_t uffd_feature_required;
+} uffd_test_case_t;
+
struct {
unsigned int pass, skip, fail, total;
} uffd_test_acct;
@@ -108,9 +168,50 @@ static int test_uffd_api(bool use_dev)
return 1;
}
+/*
+ * This function initializes the global variables. TODO: remove global
+ * vars and then remove this.
+ */
+static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type)
+{
+ map_shared = mem_type->shared;
+ uffd_test_ops = mem_type->mem_ops;
+
+ if (mem_type->mem_flag & (MEM_HUGETLB_PRIVATE | MEM_HUGETLB))
+ page_size = default_huge_page_size();
+ else
+ page_size = psize();
+
+ nr_pages = UFFD_TEST_MEM_SIZE / page_size;
+ /* TODO: remove this global var.. it's so ugly */
+ nr_cpus = 1;
+
+ return uffd_test_ctx_init(test->uffd_feature_required);
+}
+
+static bool uffd_feature_supported(uffd_test_case_t *test)
+{
+ uint64_t features;
+
+ if (uffd_get_features(&features))
+ return false;
+
+ return (features & test->uffd_feature_required) ==
+ test->uffd_feature_required;
+}
+
+uffd_test_case_t uffd_tests[] = {
+};
+
int main(int argc, char *argv[])
{
+ int n_tests = sizeof(uffd_tests) / sizeof(uffd_test_case_t);
+ int n_mems = sizeof(mem_types) / sizeof(mem_type_t);
+ uffd_test_case_t *test;
+ mem_type_t *mem_type;
+ char test_name[128];
int has_uffd;
+ int i, j;
has_uffd = test_uffd_api(false);
has_uffd |= test_uffd_api(true);
@@ -119,7 +220,31 @@ int main(int argc, char *argv[])
printf("Userfaultfd not supported or unprivileged, skip all tests\n");
exit(KSFT_SKIP);
}
+
+ for (i = 0; i < n_tests; i++) {
+ test = &uffd_tests[i];
+ for (j = 0; j < n_mems; j++) {
+ mem_type = &mem_types[j];
+ if (!(test->mem_targets & mem_type->mem_flag))
+ continue;
+ snprintf(test_name, sizeof(test_name),
+ "%s on %s", test->name, mem_type->name);
+
+ uffd_test_start(test_name);
+ if (!uffd_feature_supported(test)) {
+ uffd_test_skip("feature missing");
+ continue;
+ }
+ if (uffd_setup_environment(test, mem_type)) {
+ uffd_test_skip("memory allocation failed");
+ continue;
+ }
+ test->uffd_fn();
+ }
+ }
+
uffd_test_report();
+
return uffd_test_acct.fail ? KSFT_FAIL : KSFT_PASS;
}
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 62fcf039d6b7..dad1f62a7ecd 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -264,3 +264,30 @@ int uffd_open(unsigned int flags)
return uffd;
}
+
+int uffd_get_features(uint64_t *features)
+{
+ struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
+ /*
+ * This should by default work in most kernels; the feature list
+ * will be the same no matter what we pass in here.
+ */
+ int fd = uffd_open(UFFD_USER_MODE_ONLY);
+
+ if (fd < 0)
+ /* Maybe the kernel is older than user-only mode? */
+ fd = uffd_open(0);
+
+ if (fd < 0)
+ return fd;
+
+ if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
+ close(fd);
+ return -errno;
+ }
+
+ *features = uffdio_api.features;
+ close(fd);
+
+ return 0;
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index a67db8432855..2edad3256271 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -51,6 +51,7 @@ int uffd_open(unsigned int flags);
int uffd_register(int uffd, void *addr, uint64_t len,
bool miss, bool wp, bool minor);
int uffd_unregister(int uffd, void *addr, uint64_t len);
+int uffd_get_features(uint64_t *features);
/*
* On ppc64 this will only work with radix 2M hugepage size
--
2.39.1
Move common utility functions into uffd-common.[ch] files from the original
userfaultfd.c. This prepares for a split of userfaultfd.c into two tests:
one to only cover the old but powerful stress test, the other one covers
all the functional tests.
This movement is kind of a brute-force effort for now, with light touch-ups
but nothing should really change. There's chances to optimize more, but
let's leave that for later.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/Makefile | 2 +
tools/testing/selftests/mm/uffd-common.c | 611 ++++++++++++++++++++
tools/testing/selftests/mm/uffd-common.h | 117 ++++
tools/testing/selftests/mm/userfaultfd.c | 694 +----------------------
4 files changed, 731 insertions(+), 693 deletions(-)
create mode 100644 tools/testing/selftests/mm/uffd-common.c
create mode 100644 tools/testing/selftests/mm/uffd-common.h
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index b35f3eafde3c..9c3737285f8a 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -107,6 +107,8 @@ include ../lib.mk
$(TEST_GEN_PROGS): vm_util.c
+$(OUTPUT)/userfaultfd: uffd-common.c
+
ifeq ($(MACHINE),x86_64)
BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
BINARIES_64 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_64))
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
new file mode 100644
index 000000000000..c57757c2a36f
--- /dev/null
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -0,0 +1,611 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Userfaultfd tests util functions
+ *
+ * Copyright (C) 2015-2023 Red Hat, Inc.
+ */
+
+#include "uffd-common.h"
+
+#define BASE_PMD_ADDR ((void *)(1UL << 30))
+
+volatile bool test_uffdio_copy_eexist = true;
+unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
+char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
+int mem_fd, uffd = -1, uffd_flags, finished, *pipefd, test_type;
+bool map_shared, test_collapse, test_dev_userfaultfd;
+bool test_uffdio_wp = true, test_uffdio_minor = false;
+unsigned long long *count_verify;
+uffd_test_ops_t *uffd_test_ops;
+
+static void anon_release_pages(char *rel_area)
+{
+ if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+}
+
+static void anon_allocate_area(void **alloc_area, bool is_src)
+{
+ *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
+ MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+}
+
+static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
+{
+}
+
+static void hugetlb_release_pages(char *rel_area)
+{
+ if (!map_shared) {
+ if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+ } else {
+ if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
+ err("madvise(MADV_REMOVE) failed");
+ }
+}
+
+static void hugetlb_allocate_area(void **alloc_area, bool is_src)
+{
+ off_t size = nr_pages * page_size;
+ off_t offset = is_src ? 0 : size;
+ void *area_alias = NULL;
+ char **alloc_area_alias;
+
+ *alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
+ (map_shared ? MAP_SHARED : MAP_PRIVATE) |
+ (is_src ? 0 : MAP_NORESERVE),
+ mem_fd, offset);
+ if (*alloc_area == MAP_FAILED)
+ err("mmap of hugetlbfs file failed");
+
+ if (map_shared) {
+ area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
+ MAP_SHARED, mem_fd, offset);
+ if (area_alias == MAP_FAILED)
+ err("mmap of hugetlb file alias failed");
+ }
+
+ if (is_src) {
+ alloc_area_alias = &area_src_alias;
+ } else {
+ alloc_area_alias = &area_dst_alias;
+ }
+ if (area_alias)
+ *alloc_area_alias = area_alias;
+}
+
+static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
+{
+ if (!map_shared)
+ return;
+
+ *start = (unsigned long) area_dst_alias + offset;
+}
+
+static void shmem_release_pages(char *rel_area)
+{
+ if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
+ err("madvise(MADV_REMOVE) failed");
+}
+
+static void shmem_allocate_area(void **alloc_area, bool is_src)
+{
+ void *area_alias = NULL;
+ size_t bytes = nr_pages * page_size;
+ unsigned long offset = is_src ? 0 : bytes;
+ char *p = NULL, *p_alias = NULL;
+
+ if (test_collapse) {
+ p = BASE_PMD_ADDR;
+ if (!is_src)
+ /* src map + alias + interleaved hpages */
+ p += 2 * (bytes + hpage_size);
+ p_alias = p;
+ p_alias += bytes;
+ p_alias += hpage_size; /* Prevent src/dst VMA merge */
+ }
+
+ *alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
+ mem_fd, offset);
+ if (*alloc_area == MAP_FAILED)
+ err("mmap of memfd failed");
+ if (test_collapse && *alloc_area != p)
+ err("mmap of memfd failed at %p", p);
+
+ area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
+ mem_fd, offset);
+ if (area_alias == MAP_FAILED)
+ err("mmap of memfd alias failed");
+ if (test_collapse && area_alias != p_alias)
+ err("mmap of anonymous memory failed at %p", p_alias);
+
+ if (is_src)
+ area_src_alias = area_alias;
+ else
+ area_dst_alias = area_alias;
+}
+
+static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
+{
+ *start = (unsigned long)area_dst_alias + offset;
+}
+
+static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
+{
+ if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
+ err("Did not find expected %d number of hugepages",
+ expect_nr_hpages);
+}
+
+struct uffd_test_ops anon_uffd_test_ops = {
+ .allocate_area = anon_allocate_area,
+ .release_pages = anon_release_pages,
+ .alias_mapping = noop_alias_mapping,
+ .check_pmd_mapping = NULL,
+};
+
+struct uffd_test_ops shmem_uffd_test_ops = {
+ .allocate_area = shmem_allocate_area,
+ .release_pages = shmem_release_pages,
+ .alias_mapping = shmem_alias_mapping,
+ .check_pmd_mapping = shmem_check_pmd_mapping,
+};
+
+struct uffd_test_ops hugetlb_uffd_test_ops = {
+ .allocate_area = hugetlb_allocate_area,
+ .release_pages = hugetlb_release_pages,
+ .alias_mapping = hugetlb_alias_mapping,
+ .check_pmd_mapping = NULL,
+};
+
+void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
+{
+ int i;
+ unsigned long long miss_total = 0, wp_total = 0, minor_total = 0;
+
+ for (i = 0; i < n_cpus; i++) {
+ miss_total += stats[i].missing_faults;
+ wp_total += stats[i].wp_faults;
+ minor_total += stats[i].minor_faults;
+ }
+
+ printf("userfaults: ");
+ if (miss_total) {
+ printf("%llu missing (", miss_total);
+ for (i = 0; i < n_cpus; i++)
+ printf("%lu+", stats[i].missing_faults);
+ printf("\b) ");
+ }
+ if (wp_total) {
+ printf("%llu wp (", wp_total);
+ for (i = 0; i < n_cpus; i++)
+ printf("%lu+", stats[i].wp_faults);
+ printf("\b) ");
+ }
+ if (minor_total) {
+ printf("%llu minor (", minor_total);
+ for (i = 0; i < n_cpus; i++)
+ printf("%lu+", stats[i].minor_faults);
+ printf("\b)");
+ }
+ printf("\n");
+}
+
+static int __userfaultfd_open_dev(void)
+{
+ int fd, _uffd;
+
+ fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+ if (fd < 0)
+ errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
+
+ _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
+ if (_uffd < 0)
+ errexit(errno == ENOTTY ? KSFT_SKIP : 1,
+ "creating userfaultfd failed");
+ close(fd);
+ return _uffd;
+}
+
+void userfaultfd_open(uint64_t *features)
+{
+ struct uffdio_api uffdio_api;
+
+ if (test_dev_userfaultfd)
+ uffd = __userfaultfd_open_dev();
+ else {
+ uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
+ if (uffd < 0)
+ errexit(errno == ENOSYS ? KSFT_SKIP : 1,
+ "creating userfaultfd failed");
+ }
+ uffd_flags = fcntl(uffd, F_GETFD, NULL);
+
+ uffdio_api.api = UFFD_API;
+ uffdio_api.features = *features;
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api))
+ err("UFFDIO_API failed.\nPlease make sure to "
+ "run with either root or ptrace capability.");
+ if (uffdio_api.api != UFFD_API)
+ err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api);
+
+ *features = uffdio_api.features;
+}
+
+static inline void munmap_area(void **area)
+{
+ if (*area)
+ if (munmap(*area, nr_pages * page_size))
+ err("munmap");
+
+ *area = NULL;
+}
+
+static void uffd_test_ctx_clear(void)
+{
+ size_t i;
+
+ if (pipefd) {
+ for (i = 0; i < nr_cpus * 2; ++i) {
+ if (close(pipefd[i]))
+ err("close pipefd");
+ }
+ free(pipefd);
+ pipefd = NULL;
+ }
+
+ if (count_verify) {
+ free(count_verify);
+ count_verify = NULL;
+ }
+
+ if (uffd != -1) {
+ if (close(uffd))
+ err("close uffd");
+ uffd = -1;
+ }
+
+ munmap_area((void **)&area_src);
+ munmap_area((void **)&area_src_alias);
+ munmap_area((void **)&area_dst);
+ munmap_area((void **)&area_dst_alias);
+ munmap_area((void **)&area_remap);
+}
+
+void uffd_test_ctx_init(uint64_t features)
+{
+ unsigned long nr, cpu;
+
+ uffd_test_ctx_clear();
+
+ uffd_test_ops->allocate_area((void **)&area_src, true);
+ uffd_test_ops->allocate_area((void **)&area_dst, false);
+
+ userfaultfd_open(&features);
+
+ count_verify = malloc(nr_pages * sizeof(unsigned long long));
+ if (!count_verify)
+ err("count_verify");
+
+ for (nr = 0; nr < nr_pages; nr++) {
+ *area_mutex(area_src, nr) =
+ (pthread_mutex_t)PTHREAD_MUTEX_INITIALIZER;
+ count_verify[nr] = *area_count(area_src, nr) = 1;
+ /*
+ * In the transition between 255 to 256, powerpc will
+ * read out of order in my_bcmp and see both bytes as
+ * zero, so leave a placeholder below always non-zero
+ * after the count, to avoid my_bcmp to trigger false
+ * positives.
+ */
+ *(area_count(area_src, nr) + 1) = 1;
+ }
+
+ /*
+ * After initialization of area_src, we must explicitly release pages
+ * for area_dst to make sure it's fully empty. Otherwise we could have
+ * some area_dst pages be errornously initialized with zero pages,
+ * hence we could hit memory corruption later in the test.
+ *
+ * One example is when THP is globally enabled, above allocate_area()
+ * calls could have the two areas merged into a single VMA (as they
+ * will have the same VMA flags so they're mergeable). When we
+ * initialize the area_src above, it's possible that some part of
+ * area_dst could have been faulted in via one huge THP that will be
+ * shared between area_src and area_dst. It could cause some of the
+ * area_dst won't be trapped by missing userfaults.
+ *
+ * This release_pages() will guarantee even if that happened, we'll
+ * proactively split the thp and drop any accidentally initialized
+ * pages within area_dst.
+ */
+ uffd_test_ops->release_pages(area_dst);
+
+ pipefd = malloc(sizeof(int) * nr_cpus * 2);
+ if (!pipefd)
+ err("pipefd");
+ for (cpu = 0; cpu < nr_cpus; cpu++)
+ if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
+ err("pipe");
+}
+
+uint64_t get_expected_ioctls(uint64_t mode)
+{
+ uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
+
+ if (test_type == TEST_HUGETLB)
+ ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
+
+ if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
+ ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
+
+ if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
+ ioctls &= ~(1 << _UFFDIO_CONTINUE);
+
+ return ioctls;
+}
+
+void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
+{
+ uint64_t expected = get_expected_ioctls(mode);
+ uint64_t actual = ioctls & expected;
+
+ if (actual != expected) {
+ err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
+ expected, actual);
+ }
+}
+
+void wp_range(int ufd, __u64 start, __u64 len, bool wp)
+{
+ struct uffdio_writeprotect prms;
+
+ /* Write protection page faults */
+ prms.range.start = start;
+ prms.range.len = len;
+ /* Undo write-protect, do wakeup after that */
+ prms.mode = wp ? UFFDIO_WRITEPROTECT_MODE_WP : 0;
+
+ if (ioctl(ufd, UFFDIO_WRITEPROTECT, &prms))
+ err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
+}
+
+static void continue_range(int ufd, __u64 start, __u64 len)
+{
+ struct uffdio_continue req;
+ int ret;
+
+ req.range.start = start;
+ req.range.len = len;
+ req.mode = 0;
+ if (test_uffdio_wp)
+ req.mode |= UFFDIO_CONTINUE_MODE_WP;
+
+ if (ioctl(ufd, UFFDIO_CONTINUE, &req))
+ err("UFFDIO_CONTINUE failed for address 0x%" PRIx64,
+ (uint64_t)start);
+
+ /*
+ * Error handling within the kernel for continue is subtly different
+ * from copy or zeropage, so it may be a source of bugs. Trigger an
+ * error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG.
+ */
+ req.mapped = 0;
+ ret = ioctl(ufd, UFFDIO_CONTINUE, &req);
+ if (ret >= 0 || req.mapped != -EEXIST)
+ err("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64,
+ ret, (int64_t) req.mapped);
+}
+
+int uffd_read_msg(int ufd, struct uffd_msg *msg)
+{
+ int ret = read(uffd, msg, sizeof(*msg));
+
+ if (ret != sizeof(*msg)) {
+ if (ret < 0) {
+ if (errno == EAGAIN || errno == EINTR)
+ return 1;
+ err("blocking read error");
+ } else {
+ err("short read");
+ }
+ }
+
+ return 0;
+}
+
+void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
+{
+ unsigned long offset;
+
+ if (msg->event != UFFD_EVENT_PAGEFAULT)
+ err("unexpected msg event %u", msg->event);
+
+ if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) {
+ /* Write protect page faults */
+ wp_range(uffd, msg->arg.pagefault.address, page_size, false);
+ stats->wp_faults++;
+ } else if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_MINOR) {
+ uint8_t *area;
+ int b;
+
+ /*
+ * Minor page faults
+ *
+ * To prove we can modify the original range for testing
+ * purposes, we're going to bit flip this range before
+ * continuing.
+ *
+ * Note that this requires all minor page fault tests operate on
+ * area_dst (non-UFFD-registered) and area_dst_alias
+ * (UFFD-registered).
+ */
+
+ area = (uint8_t *)(area_dst +
+ ((char *)msg->arg.pagefault.address -
+ area_dst_alias));
+ for (b = 0; b < page_size; ++b)
+ area[b] = ~area[b];
+ continue_range(uffd, msg->arg.pagefault.address, page_size);
+ stats->minor_faults++;
+ } else {
+ /*
+ * Missing page faults.
+ *
+ * Here we force a write check for each of the missing mode
+ * faults. It's guaranteed because the only threads that
+ * will trigger uffd faults are the locking threads, and
+ * their first instruction to touch the missing page will
+ * always be pthread_mutex_lock().
+ *
+ * Note that here we relied on an NPTL glibc impl detail to
+ * always read the lock type at the entry of the lock op
+ * (pthread_mutex_t.__data.__type, offset 0x10) before
+ * doing any locking operations to guarantee that. It's
+ * actually not good to rely on this impl detail because
+ * logically a pthread-compatible lib can implement the
+ * locks without types and we can fail when linking with
+ * them. However since we used to find bugs with this
+ * strict check we still keep it around. Hopefully this
+ * could be a good hint when it fails again. If one day
+ * it'll break on some other impl of glibc we'll revisit.
+ */
+ if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE)
+ err("unexpected write fault");
+
+ offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
+ offset &= ~(page_size-1);
+
+ if (copy_page(uffd, offset))
+ stats->missing_faults++;
+ }
+}
+
+void *uffd_poll_thread(void *arg)
+{
+ struct uffd_stats *stats = (struct uffd_stats *)arg;
+ unsigned long cpu = stats->cpu;
+ struct pollfd pollfd[2];
+ struct uffd_msg msg;
+ struct uffdio_register uffd_reg;
+ int ret;
+ char tmp_chr;
+
+ pollfd[0].fd = uffd;
+ pollfd[0].events = POLLIN;
+ pollfd[1].fd = pipefd[cpu*2];
+ pollfd[1].events = POLLIN;
+
+ for (;;) {
+ ret = poll(pollfd, 2, -1);
+ if (ret <= 0) {
+ if (errno == EINTR || errno == EAGAIN)
+ continue;
+ err("poll error: %d", ret);
+ }
+ if (pollfd[1].revents) {
+ if (!(pollfd[1].revents & POLLIN))
+ err("pollfd[1].revents %d", pollfd[1].revents);
+ if (read(pollfd[1].fd, &tmp_chr, 1) != 1)
+ err("read pipefd error");
+ break;
+ }
+ if (!(pollfd[0].revents & POLLIN))
+ err("pollfd[0].revents %d", pollfd[0].revents);
+ if (uffd_read_msg(uffd, &msg))
+ continue;
+ switch (msg.event) {
+ default:
+ err("unexpected msg event %u\n", msg.event);
+ break;
+ case UFFD_EVENT_PAGEFAULT:
+ uffd_handle_page_fault(&msg, stats);
+ break;
+ case UFFD_EVENT_FORK:
+ close(uffd);
+ uffd = msg.arg.fork.ufd;
+ pollfd[0].fd = uffd;
+ break;
+ case UFFD_EVENT_REMOVE:
+ uffd_reg.range.start = msg.arg.remove.start;
+ uffd_reg.range.len = msg.arg.remove.end -
+ msg.arg.remove.start;
+ if (ioctl(uffd, UFFDIO_UNREGISTER, &uffd_reg.range))
+ err("remove failure");
+ break;
+ case UFFD_EVENT_REMAP:
+ area_remap = area_dst; /* save for later unmap */
+ area_dst = (char *)(unsigned long)msg.arg.remap.to;
+ break;
+ }
+ }
+
+ return NULL;
+}
+
+static void retry_copy_page(int ufd, struct uffdio_copy *uffdio_copy,
+ unsigned long offset)
+{
+ uffd_test_ops->alias_mapping(&uffdio_copy->dst,
+ uffdio_copy->len,
+ offset);
+ if (ioctl(ufd, UFFDIO_COPY, uffdio_copy)) {
+ /* real retval in ufdio_copy.copy */
+ if (uffdio_copy->copy != -EEXIST)
+ err("UFFDIO_COPY retry error: %"PRId64,
+ (int64_t)uffdio_copy->copy);
+ } else {
+ err("UFFDIO_COPY retry unexpected: %"PRId64,
+ (int64_t)uffdio_copy->copy);
+ }
+}
+
+static void wake_range(int ufd, unsigned long addr, unsigned long len)
+{
+ struct uffdio_range uffdio_wake;
+
+ uffdio_wake.start = addr;
+ uffdio_wake.len = len;
+
+ if (ioctl(ufd, UFFDIO_WAKE, &uffdio_wake))
+ fprintf(stderr, "error waking %lu\n",
+ addr), exit(1);
+}
+
+int __copy_page(int ufd, unsigned long offset, bool retry)
+{
+ struct uffdio_copy uffdio_copy;
+
+ if (offset >= nr_pages * page_size)
+ err("unexpected offset %lu\n", offset);
+ uffdio_copy.dst = (unsigned long) area_dst + offset;
+ uffdio_copy.src = (unsigned long) area_src + offset;
+ uffdio_copy.len = page_size;
+ if (test_uffdio_wp)
+ uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
+ else
+ uffdio_copy.mode = 0;
+ uffdio_copy.copy = 0;
+ if (ioctl(ufd, UFFDIO_COPY, &uffdio_copy)) {
+ /* real retval in ufdio_copy.copy */
+ if (uffdio_copy.copy != -EEXIST)
+ err("UFFDIO_COPY error: %"PRId64,
+ (int64_t)uffdio_copy.copy);
+ wake_range(ufd, uffdio_copy.dst, page_size);
+ } else if (uffdio_copy.copy != page_size) {
+ err("UFFDIO_COPY error: %"PRId64, (int64_t)uffdio_copy.copy);
+ } else {
+ if (test_uffdio_copy_eexist && retry) {
+ test_uffdio_copy_eexist = false;
+ retry_copy_page(ufd, &uffdio_copy, offset);
+ }
+ return 1;
+ }
+ return 0;
+}
+
+int copy_page(int ufd, unsigned long offset)
+{
+ return __copy_page(ufd, offset, false);
+}
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
new file mode 100644
index 000000000000..d9430cfdcb19
--- /dev/null
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Userfaultfd tests common header
+ *
+ * Copyright (C) 2015-2023 Red Hat, Inc.
+ */
+#ifndef __UFFD_COMMON_H__
+#define __UFFD_COMMON_H__
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <errno.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <time.h>
+#include <signal.h>
+#include <poll.h>
+#include <string.h>
+#include <linux/mman.h>
+#include <sys/mman.h>
+#include <sys/syscall.h>
+#include <sys/ioctl.h>
+#include <sys/wait.h>
+#include <pthread.h>
+#include <linux/userfaultfd.h>
+#include <setjmp.h>
+#include <stdbool.h>
+#include <assert.h>
+#include <inttypes.h>
+#include <stdint.h>
+#include <sys/random.h>
+
+#include "../kselftest.h"
+#include "vm_util.h"
+
+#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)
+
+#define _err(fmt, ...) \
+ do { \
+ int ret = errno; \
+ fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \
+ fprintf(stderr, " (errno=%d, @%s:%d)\n", \
+ ret, __FILE__, __LINE__); \
+ } while (0)
+
+#define errexit(exitcode, fmt, ...) \
+ do { \
+ _err(fmt, ##__VA_ARGS__); \
+ exit(exitcode); \
+ } while (0)
+
+#define err(fmt, ...) errexit(1, fmt, ##__VA_ARGS__)
+
+/* pthread_mutex_t starts at page offset 0 */
+#define area_mutex(___area, ___nr) \
+ ((pthread_mutex_t *) ((___area) + (___nr)*page_size))
+/*
+ * count is placed in the page after pthread_mutex_t naturally aligned
+ * to avoid non alignment faults on non-x86 archs.
+ */
+#define area_count(___area, ___nr) \
+ ((volatile unsigned long long *) ((unsigned long) \
+ ((___area) + (___nr)*page_size + \
+ sizeof(pthread_mutex_t) + \
+ sizeof(unsigned long long) - 1) & \
+ ~(unsigned long)(sizeof(unsigned long long) \
+ - 1)))
+
+/* Userfaultfd test statistics */
+struct uffd_stats {
+ int cpu;
+ unsigned long missing_faults;
+ unsigned long wp_faults;
+ unsigned long minor_faults;
+};
+
+struct uffd_test_ops {
+ void (*allocate_area)(void **alloc_area, bool is_src);
+ void (*release_pages)(char *rel_area);
+ void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
+ void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
+};
+typedef struct uffd_test_ops uffd_test_ops_t;
+
+extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
+extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
+extern int mem_fd, uffd, uffd_flags, finished, *pipefd, test_type;
+extern bool map_shared, test_collapse, test_dev_userfaultfd;
+extern bool test_uffdio_wp, test_uffdio_minor;
+extern unsigned long long *count_verify;
+extern volatile bool test_uffdio_copy_eexist;
+
+extern uffd_test_ops_t anon_uffd_test_ops;
+extern uffd_test_ops_t shmem_uffd_test_ops;
+extern uffd_test_ops_t hugetlb_uffd_test_ops;
+extern uffd_test_ops_t *uffd_test_ops;
+
+void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
+void uffd_test_ctx_init(uint64_t features);
+void userfaultfd_open(uint64_t *features);
+uint64_t get_expected_ioctls(uint64_t mode);
+void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls);
+int uffd_read_msg(int ufd, struct uffd_msg *msg);
+void wp_range(int ufd, __u64 start, __u64 len, bool wp);
+void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
+int __copy_page(int ufd, unsigned long offset, bool retry);
+int copy_page(int ufd, unsigned long offset);
+void *uffd_poll_thread(void *arg);
+
+#define TEST_ANON 1
+#define TEST_HUGETLB 2
+#define TEST_SHMEM 3
+
+#endif
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index 3487ec0bfcc8..c68a9aeefc41 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -34,96 +34,20 @@
* transfer (UFFDIO_COPY).
*/
-#define _GNU_SOURCE
-#include <stdio.h>
-#include <errno.h>
-#include <unistd.h>
-#include <stdlib.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <time.h>
-#include <signal.h>
-#include <poll.h>
-#include <string.h>
-#include <linux/mman.h>
-#include <sys/mman.h>
-#include <sys/syscall.h>
-#include <sys/ioctl.h>
-#include <sys/wait.h>
-#include <pthread.h>
-#include <linux/userfaultfd.h>
-#include <setjmp.h>
-#include <stdbool.h>
-#include <assert.h>
-#include <inttypes.h>
-#include <stdint.h>
-#include <sys/random.h>
-
-#include "../kselftest.h"
-#include "vm_util.h"
+#include "uffd-common.h"
#ifdef __NR_userfaultfd
-static unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
-
#define BOUNCE_RANDOM (1<<0)
#define BOUNCE_RACINGFAULTS (1<<1)
#define BOUNCE_VERIFY (1<<2)
#define BOUNCE_POLL (1<<3)
static int bounces;
-#define TEST_ANON 1
-#define TEST_HUGETLB 2
-#define TEST_SHMEM 3
-static int test_type;
-
-#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)
-
-#define BASE_PMD_ADDR ((void *)(1UL << 30))
-
-/* test using /dev/userfaultfd, instead of userfaultfd(2) */
-static bool test_dev_userfaultfd;
-
/* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */
#define ALARM_INTERVAL_SECS 10
-static volatile bool test_uffdio_copy_eexist = true;
-/* Whether to test uffd write-protection */
-static bool test_uffdio_wp = true;
-/* Whether to test uffd minor faults */
-static bool test_uffdio_minor = false;
-static bool map_shared;
-static int mem_fd;
-static unsigned long long *count_verify;
-static int uffd = -1;
-static int uffd_flags, finished, *pipefd;
-static char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
static char *zeropage;
pthread_attr_t attr;
-static bool test_collapse;
-
-/* Userfaultfd test statistics */
-struct uffd_stats {
- int cpu;
- unsigned long missing_faults;
- unsigned long wp_faults;
- unsigned long minor_faults;
-};
-
-/* pthread_mutex_t starts at page offset 0 */
-#define area_mutex(___area, ___nr) \
- ((pthread_mutex_t *) ((___area) + (___nr)*page_size))
-/*
- * count is placed in the page after pthread_mutex_t naturally aligned
- * to avoid non alignment faults on non-x86 archs.
- */
-#define area_count(___area, ___nr) \
- ((volatile unsigned long long *) ((unsigned long) \
- ((___area) + (___nr)*page_size + \
- sizeof(pthread_mutex_t) + \
- sizeof(unsigned long long) - 1) & \
- ~(unsigned long)(sizeof(unsigned long long) \
- - 1)))
#define swap(a, b) \
do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
@@ -166,22 +90,6 @@ static void usage(void)
exit(1);
}
-#define _err(fmt, ...) \
- do { \
- int ret = errno; \
- fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \
- fprintf(stderr, " (errno=%d, line=%d)\n", \
- ret, __LINE__); \
- } while (0)
-
-#define errexit(exitcode, fmt, ...) \
- do { \
- _err(fmt, ##__VA_ARGS__); \
- exit(exitcode); \
- } while (0)
-
-#define err(fmt, ...) errexit(1, fmt, ##__VA_ARGS__)
-
static void uffd_stats_reset(struct uffd_stats *uffd_stats,
unsigned long n_cpus)
{
@@ -195,189 +103,6 @@ static void uffd_stats_reset(struct uffd_stats *uffd_stats,
}
}
-static void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
-{
- int i;
- unsigned long long miss_total = 0, wp_total = 0, minor_total = 0;
-
- for (i = 0; i < n_cpus; i++) {
- miss_total += stats[i].missing_faults;
- wp_total += stats[i].wp_faults;
- minor_total += stats[i].minor_faults;
- }
-
- printf("userfaults: ");
- if (miss_total) {
- printf("%llu missing (", miss_total);
- for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].missing_faults);
- printf("\b) ");
- }
- if (wp_total) {
- printf("%llu wp (", wp_total);
- for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].wp_faults);
- printf("\b) ");
- }
- if (minor_total) {
- printf("%llu minor (", minor_total);
- for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].minor_faults);
- printf("\b)");
- }
- printf("\n");
-}
-
-static void anon_release_pages(char *rel_area)
-{
- if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
-}
-
-static void anon_allocate_area(void **alloc_area, bool is_src)
-{
- *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
- MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
-}
-
-static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
-{
-}
-
-static void hugetlb_release_pages(char *rel_area)
-{
- if (!map_shared) {
- if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
- } else {
- if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
- err("madvise(MADV_REMOVE) failed");
- }
-}
-
-static void hugetlb_allocate_area(void **alloc_area, bool is_src)
-{
- off_t size = nr_pages * page_size;
- off_t offset = is_src ? 0 : size;
- void *area_alias = NULL;
- char **alloc_area_alias;
-
- *alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
- (map_shared ? MAP_SHARED : MAP_PRIVATE) |
- (is_src ? 0 : MAP_NORESERVE),
- mem_fd, offset);
- if (*alloc_area == MAP_FAILED)
- err("mmap of hugetlbfs file failed");
-
- if (map_shared) {
- area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
- MAP_SHARED, mem_fd, offset);
- if (area_alias == MAP_FAILED)
- err("mmap of hugetlb file alias failed");
- }
-
- if (is_src) {
- alloc_area_alias = &area_src_alias;
- } else {
- alloc_area_alias = &area_dst_alias;
- }
- if (area_alias)
- *alloc_area_alias = area_alias;
-}
-
-static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
-{
- if (!map_shared)
- return;
-
- *start = (unsigned long) area_dst_alias + offset;
-}
-
-static void shmem_release_pages(char *rel_area)
-{
- if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
- err("madvise(MADV_REMOVE) failed");
-}
-
-static void shmem_allocate_area(void **alloc_area, bool is_src)
-{
- void *area_alias = NULL;
- size_t bytes = nr_pages * page_size;
- unsigned long offset = is_src ? 0 : bytes;
- char *p = NULL, *p_alias = NULL;
-
- if (test_collapse) {
- p = BASE_PMD_ADDR;
- if (!is_src)
- /* src map + alias + interleaved hpages */
- p += 2 * (bytes + hpage_size);
- p_alias = p;
- p_alias += bytes;
- p_alias += hpage_size; /* Prevent src/dst VMA merge */
- }
-
- *alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
- mem_fd, offset);
- if (*alloc_area == MAP_FAILED)
- err("mmap of memfd failed");
- if (test_collapse && *alloc_area != p)
- err("mmap of memfd failed at %p", p);
-
- area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
- mem_fd, offset);
- if (area_alias == MAP_FAILED)
- err("mmap of memfd alias failed");
- if (test_collapse && area_alias != p_alias)
- err("mmap of anonymous memory failed at %p", p_alias);
-
- if (is_src)
- area_src_alias = area_alias;
- else
- area_dst_alias = area_alias;
-}
-
-static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
-{
- *start = (unsigned long)area_dst_alias + offset;
-}
-
-static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
-{
- if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
- err("Did not find expected %d number of hugepages",
- expect_nr_hpages);
-}
-
-struct uffd_test_ops {
- void (*allocate_area)(void **alloc_area, bool is_src);
- void (*release_pages)(char *rel_area);
- void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
- void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
-};
-
-static struct uffd_test_ops anon_uffd_test_ops = {
- .allocate_area = anon_allocate_area,
- .release_pages = anon_release_pages,
- .alias_mapping = noop_alias_mapping,
- .check_pmd_mapping = NULL,
-};
-
-static struct uffd_test_ops shmem_uffd_test_ops = {
- .allocate_area = shmem_allocate_area,
- .release_pages = shmem_release_pages,
- .alias_mapping = shmem_alias_mapping,
- .check_pmd_mapping = shmem_check_pmd_mapping,
-};
-
-static struct uffd_test_ops hugetlb_uffd_test_ops = {
- .allocate_area = hugetlb_allocate_area,
- .release_pages = hugetlb_release_pages,
- .alias_mapping = hugetlb_alias_mapping,
- .check_pmd_mapping = NULL,
-};
-
-static struct uffd_test_ops *uffd_test_ops;
-
static inline uint64_t uffd_minor_feature(void)
{
if (test_type == TEST_HUGETLB && map_shared)
@@ -388,171 +113,6 @@ static inline uint64_t uffd_minor_feature(void)
return 0;
}
-static uint64_t get_expected_ioctls(uint64_t mode)
-{
- uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
-
- if (test_type == TEST_HUGETLB)
- ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
-
- if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
- ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
-
- if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
- ioctls &= ~(1 << _UFFDIO_CONTINUE);
-
- return ioctls;
-}
-
-static void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
-{
- uint64_t expected = get_expected_ioctls(mode);
- uint64_t actual = ioctls & expected;
-
- if (actual != expected) {
- err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
- expected, actual);
- }
-}
-
-static int __userfaultfd_open_dev(void)
-{
- int fd, _uffd;
-
- fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
- if (fd < 0)
- errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
-
- _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
- if (_uffd < 0)
- errexit(errno == ENOTTY ? KSFT_SKIP : 1,
- "creating userfaultfd failed");
- close(fd);
- return _uffd;
-}
-
-static void userfaultfd_open(uint64_t *features)
-{
- struct uffdio_api uffdio_api;
-
- if (test_dev_userfaultfd)
- uffd = __userfaultfd_open_dev();
- else {
- uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
- if (uffd < 0)
- errexit(errno == ENOSYS ? KSFT_SKIP : 1,
- "creating userfaultfd failed");
- }
- uffd_flags = fcntl(uffd, F_GETFD, NULL);
-
- uffdio_api.api = UFFD_API;
- uffdio_api.features = *features;
- if (ioctl(uffd, UFFDIO_API, &uffdio_api))
- err("UFFDIO_API failed.\nPlease make sure to "
- "run with either root or ptrace capability.");
- if (uffdio_api.api != UFFD_API)
- err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api);
-
- *features = uffdio_api.features;
-}
-
-static inline void munmap_area(void **area)
-{
- if (*area)
- if (munmap(*area, nr_pages * page_size))
- err("munmap");
-
- *area = NULL;
-}
-
-static void uffd_test_ctx_clear(void)
-{
- size_t i;
-
- if (pipefd) {
- for (i = 0; i < nr_cpus * 2; ++i) {
- if (close(pipefd[i]))
- err("close pipefd");
- }
- free(pipefd);
- pipefd = NULL;
- }
-
- if (count_verify) {
- free(count_verify);
- count_verify = NULL;
- }
-
- if (uffd != -1) {
- if (close(uffd))
- err("close uffd");
- uffd = -1;
- }
-
- munmap_area((void **)&area_src);
- munmap_area((void **)&area_src_alias);
- munmap_area((void **)&area_dst);
- munmap_area((void **)&area_dst_alias);
- munmap_area((void **)&area_remap);
-}
-
-static void uffd_test_ctx_init(uint64_t features)
-{
- unsigned long nr, cpu;
-
- uffd_test_ctx_clear();
-
- uffd_test_ops->allocate_area((void **)&area_src, true);
- uffd_test_ops->allocate_area((void **)&area_dst, false);
-
- userfaultfd_open(&features);
-
- count_verify = malloc(nr_pages * sizeof(unsigned long long));
- if (!count_verify)
- err("count_verify");
-
- for (nr = 0; nr < nr_pages; nr++) {
- *area_mutex(area_src, nr) =
- (pthread_mutex_t)PTHREAD_MUTEX_INITIALIZER;
- count_verify[nr] = *area_count(area_src, nr) = 1;
- /*
- * In the transition between 255 to 256, powerpc will
- * read out of order in my_bcmp and see both bytes as
- * zero, so leave a placeholder below always non-zero
- * after the count, to avoid my_bcmp to trigger false
- * positives.
- */
- *(area_count(area_src, nr) + 1) = 1;
- }
-
- /*
- * After initialization of area_src, we must explicitly release pages
- * for area_dst to make sure it's fully empty. Otherwise we could have
- * some area_dst pages be errornously initialized with zero pages,
- * hence we could hit memory corruption later in the test.
- *
- * One example is when THP is globally enabled, above allocate_area()
- * calls could have the two areas merged into a single VMA (as they
- * will have the same VMA flags so they're mergeable). When we
- * initialize the area_src above, it's possible that some part of
- * area_dst could have been faulted in via one huge THP that will be
- * shared between area_src and area_dst. It could cause some of the
- * area_dst won't be trapped by missing userfaults.
- *
- * This release_pages() will guarantee even if that happened, we'll
- * proactively split the thp and drop any accidentally initialized
- * pages within area_dst.
- */
- uffd_test_ops->release_pages(area_dst);
-
- pipefd = malloc(sizeof(int) * nr_cpus * 2);
- if (!pipefd)
- err("pipefd");
- for (cpu = 0; cpu < nr_cpus; cpu++)
- if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
- err("pipe");
-}
-
static int my_bcmp(char *str1, char *str2, size_t n)
{
unsigned long i;
@@ -562,47 +122,6 @@ static int my_bcmp(char *str1, char *str2, size_t n)
return 0;
}
-static void wp_range(int ufd, __u64 start, __u64 len, bool wp)
-{
- struct uffdio_writeprotect prms;
-
- /* Write protection page faults */
- prms.range.start = start;
- prms.range.len = len;
- /* Undo write-protect, do wakeup after that */
- prms.mode = wp ? UFFDIO_WRITEPROTECT_MODE_WP : 0;
-
- if (ioctl(ufd, UFFDIO_WRITEPROTECT, &prms))
- err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
-}
-
-static void continue_range(int ufd, __u64 start, __u64 len)
-{
- struct uffdio_continue req;
- int ret;
-
- req.range.start = start;
- req.range.len = len;
- req.mode = 0;
- if (test_uffdio_wp)
- req.mode |= UFFDIO_CONTINUE_MODE_WP;
-
- if (ioctl(ufd, UFFDIO_CONTINUE, &req))
- err("UFFDIO_CONTINUE failed for address 0x%" PRIx64,
- (uint64_t)start);
-
- /*
- * Error handling within the kernel for continue is subtly different
- * from copy or zeropage, so it may be a source of bugs. Trigger an
- * error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG.
- */
- req.mapped = 0;
- ret = ioctl(ufd, UFFDIO_CONTINUE, &req);
- if (ret >= 0 || req.mapped != -EEXIST)
- err("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64,
- ret, (int64_t) req.mapped);
-}
-
static void *locking_thread(void *arg)
{
unsigned long cpu = (unsigned long) arg;
@@ -635,222 +154,11 @@ static void *locking_thread(void *arg)
return NULL;
}
-static void retry_copy_page(int ufd, struct uffdio_copy *uffdio_copy,
- unsigned long offset)
-{
- uffd_test_ops->alias_mapping(&uffdio_copy->dst,
- uffdio_copy->len,
- offset);
- if (ioctl(ufd, UFFDIO_COPY, uffdio_copy)) {
- /* real retval in ufdio_copy.copy */
- if (uffdio_copy->copy != -EEXIST)
- err("UFFDIO_COPY retry error: %"PRId64,
- (int64_t)uffdio_copy->copy);
- } else {
- err("UFFDIO_COPY retry unexpected: %"PRId64,
- (int64_t)uffdio_copy->copy);
- }
-}
-
-static void wake_range(int ufd, unsigned long addr, unsigned long len)
-{
- struct uffdio_range uffdio_wake;
-
- uffdio_wake.start = addr;
- uffdio_wake.len = len;
-
- if (ioctl(ufd, UFFDIO_WAKE, &uffdio_wake))
- fprintf(stderr, "error waking %lu\n",
- addr), exit(1);
-}
-
-static int __copy_page(int ufd, unsigned long offset, bool retry)
-{
- struct uffdio_copy uffdio_copy;
-
- if (offset >= nr_pages * page_size)
- err("unexpected offset %lu\n", offset);
- uffdio_copy.dst = (unsigned long) area_dst + offset;
- uffdio_copy.src = (unsigned long) area_src + offset;
- uffdio_copy.len = page_size;
- if (test_uffdio_wp)
- uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
- else
- uffdio_copy.mode = 0;
- uffdio_copy.copy = 0;
- if (ioctl(ufd, UFFDIO_COPY, &uffdio_copy)) {
- /* real retval in ufdio_copy.copy */
- if (uffdio_copy.copy != -EEXIST)
- err("UFFDIO_COPY error: %"PRId64,
- (int64_t)uffdio_copy.copy);
- wake_range(ufd, uffdio_copy.dst, page_size);
- } else if (uffdio_copy.copy != page_size) {
- err("UFFDIO_COPY error: %"PRId64, (int64_t)uffdio_copy.copy);
- } else {
- if (test_uffdio_copy_eexist && retry) {
- test_uffdio_copy_eexist = false;
- retry_copy_page(ufd, &uffdio_copy, offset);
- }
- return 1;
- }
- return 0;
-}
-
static int copy_page_retry(int ufd, unsigned long offset)
{
return __copy_page(ufd, offset, true);
}
-static int copy_page(int ufd, unsigned long offset)
-{
- return __copy_page(ufd, offset, false);
-}
-
-static int uffd_read_msg(int ufd, struct uffd_msg *msg)
-{
- int ret = read(uffd, msg, sizeof(*msg));
-
- if (ret != sizeof(*msg)) {
- if (ret < 0) {
- if (errno == EAGAIN || errno == EINTR)
- return 1;
- err("blocking read error");
- } else {
- err("short read");
- }
- }
-
- return 0;
-}
-
-static void uffd_handle_page_fault(struct uffd_msg *msg,
- struct uffd_stats *stats)
-{
- unsigned long offset;
-
- if (msg->event != UFFD_EVENT_PAGEFAULT)
- err("unexpected msg event %u", msg->event);
-
- if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) {
- /* Write protect page faults */
- wp_range(uffd, msg->arg.pagefault.address, page_size, false);
- stats->wp_faults++;
- } else if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_MINOR) {
- uint8_t *area;
- int b;
-
- /*
- * Minor page faults
- *
- * To prove we can modify the original range for testing
- * purposes, we're going to bit flip this range before
- * continuing.
- *
- * Note that this requires all minor page fault tests operate on
- * area_dst (non-UFFD-registered) and area_dst_alias
- * (UFFD-registered).
- */
-
- area = (uint8_t *)(area_dst +
- ((char *)msg->arg.pagefault.address -
- area_dst_alias));
- for (b = 0; b < page_size; ++b)
- area[b] = ~area[b];
- continue_range(uffd, msg->arg.pagefault.address, page_size);
- stats->minor_faults++;
- } else {
- /*
- * Missing page faults.
- *
- * Here we force a write check for each of the missing mode
- * faults. It's guaranteed because the only threads that
- * will trigger uffd faults are the locking threads, and
- * their first instruction to touch the missing page will
- * always be pthread_mutex_lock().
- *
- * Note that here we relied on an NPTL glibc impl detail to
- * always read the lock type at the entry of the lock op
- * (pthread_mutex_t.__data.__type, offset 0x10) before
- * doing any locking operations to guarantee that. It's
- * actually not good to rely on this impl detail because
- * logically a pthread-compatible lib can implement the
- * locks without types and we can fail when linking with
- * them. However since we used to find bugs with this
- * strict check we still keep it around. Hopefully this
- * could be a good hint when it fails again. If one day
- * it'll break on some other impl of glibc we'll revisit.
- */
- if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE)
- err("unexpected write fault");
-
- offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
- offset &= ~(page_size-1);
-
- if (copy_page(uffd, offset))
- stats->missing_faults++;
- }
-}
-
-static void *uffd_poll_thread(void *arg)
-{
- struct uffd_stats *stats = (struct uffd_stats *)arg;
- unsigned long cpu = stats->cpu;
- struct pollfd pollfd[2];
- struct uffd_msg msg;
- struct uffdio_register uffd_reg;
- int ret;
- char tmp_chr;
-
- pollfd[0].fd = uffd;
- pollfd[0].events = POLLIN;
- pollfd[1].fd = pipefd[cpu*2];
- pollfd[1].events = POLLIN;
-
- for (;;) {
- ret = poll(pollfd, 2, -1);
- if (ret <= 0) {
- if (errno == EINTR || errno == EAGAIN)
- continue;
- err("poll error: %d", ret);
- }
- if (pollfd[1].revents & POLLIN) {
- if (read(pollfd[1].fd, &tmp_chr, 1) != 1)
- err("read pipefd error");
- break;
- }
- if (!(pollfd[0].revents & POLLIN))
- err("pollfd[0].revents %d", pollfd[0].revents);
- if (uffd_read_msg(uffd, &msg))
- continue;
- switch (msg.event) {
- default:
- err("unexpected msg event %u\n", msg.event);
- break;
- case UFFD_EVENT_PAGEFAULT:
- uffd_handle_page_fault(&msg, stats);
- break;
- case UFFD_EVENT_FORK:
- close(uffd);
- uffd = msg.arg.fork.ufd;
- pollfd[0].fd = uffd;
- break;
- case UFFD_EVENT_REMOVE:
- uffd_reg.range.start = msg.arg.remove.start;
- uffd_reg.range.len = msg.arg.remove.end -
- msg.arg.remove.start;
- if (ioctl(uffd, UFFDIO_UNREGISTER, &uffd_reg.range))
- err("remove failure");
- break;
- case UFFD_EVENT_REMAP:
- area_remap = area_dst; /* save for later unmap */
- area_dst = (char *)(unsigned long)msg.arg.remap.to;
- break;
- }
- }
-
- return NULL;
-}
-
pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;
static void *uffd_read_thread(void *arg)
--
2.39.1
Make the handler optionally apply WP bit when resolving page faults for
either missing or minor page faults. This move towards removing global
test_uffdio_wp outside of the common code.
For this, slightly abuse uffd_stats to keep one more parameter on whether
we'd like to resolve page faults with WP bit set. Note that only the name
is abused, it'll be better to be called uffd_args or similar but let's not
bother for now.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-common.c | 17 +++++++++--------
tools/testing/selftests/mm/uffd-common.h | 6 ++++--
tools/testing/selftests/mm/uffd-stress.c | 16 ++++++++++------
3 files changed, 23 insertions(+), 16 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 025e40ffc7bf..92b7e00efa8a 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -353,7 +353,7 @@ void wp_range(int ufd, __u64 start, __u64 len, bool wp)
err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
}
-static void continue_range(int ufd, __u64 start, __u64 len)
+static void continue_range(int ufd, __u64 start, __u64 len, bool wp)
{
struct uffdio_continue req;
int ret;
@@ -361,7 +361,7 @@ static void continue_range(int ufd, __u64 start, __u64 len)
req.range.start = start;
req.range.len = len;
req.mode = 0;
- if (test_uffdio_wp)
+ if (wp)
req.mode |= UFFDIO_CONTINUE_MODE_WP;
if (ioctl(ufd, UFFDIO_CONTINUE, &req))
@@ -429,7 +429,8 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
area_dst_alias));
for (b = 0; b < page_size; ++b)
area[b] = ~area[b];
- continue_range(uffd, msg->arg.pagefault.address, page_size);
+ continue_range(uffd, msg->arg.pagefault.address, page_size,
+ stats->apply_wp);
stats->minor_faults++;
} else {
/*
@@ -459,7 +460,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
offset &= ~(page_size-1);
- if (copy_page(uffd, offset))
+ if (copy_page(uffd, offset, stats->apply_wp))
stats->missing_faults++;
}
}
@@ -555,7 +556,7 @@ static void wake_range(int ufd, unsigned long addr, unsigned long len)
addr), exit(1);
}
-int __copy_page(int ufd, unsigned long offset, bool retry)
+int __copy_page(int ufd, unsigned long offset, bool retry, bool wp)
{
struct uffdio_copy uffdio_copy;
@@ -564,7 +565,7 @@ int __copy_page(int ufd, unsigned long offset, bool retry)
uffdio_copy.dst = (unsigned long) area_dst + offset;
uffdio_copy.src = (unsigned long) area_src + offset;
uffdio_copy.len = page_size;
- if (test_uffdio_wp)
+ if (wp)
uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
else
uffdio_copy.mode = 0;
@@ -587,8 +588,8 @@ int __copy_page(int ufd, unsigned long offset, bool retry)
return 0;
}
-int copy_page(int ufd, unsigned long offset)
+int copy_page(int ufd, unsigned long offset, bool wp)
{
- return __copy_page(ufd, offset, false);
+ return __copy_page(ufd, offset, false, wp);
}
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 47565b2f2dee..f4bc73ce3b48 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -72,6 +72,8 @@
/* Userfaultfd test statistics */
struct uffd_stats {
int cpu;
+ /* Whether apply wr-protects when installing pages */
+ bool apply_wp;
unsigned long missing_faults;
unsigned long wp_faults;
unsigned long minor_faults;
@@ -104,8 +106,8 @@ void userfaultfd_open(uint64_t *features);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
-int __copy_page(int ufd, unsigned long offset, bool retry);
-int copy_page(int ufd, unsigned long offset);
+int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
+int copy_page(int ufd, unsigned long offset, bool wp);
void *uffd_poll_thread(void *arg);
#define TEST_ANON 1
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 54fc9b4ffa3c..70cb0619354e 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -97,6 +97,7 @@ static void uffd_stats_reset(struct uffd_stats *uffd_stats,
for (i = 0; i < n_cpus; i++) {
uffd_stats[i].cpu = i;
+ uffd_stats[i].apply_wp = test_uffdio_wp;
uffd_stats[i].missing_faults = 0;
uffd_stats[i].wp_faults = 0;
uffd_stats[i].minor_faults = 0;
@@ -156,7 +157,7 @@ static void *locking_thread(void *arg)
static int copy_page_retry(int ufd, unsigned long offset)
{
- return __copy_page(ufd, offset, true);
+ return __copy_page(ufd, offset, true, test_uffdio_wp);
}
pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;
@@ -309,7 +310,7 @@ static void sighndl(int sig, siginfo_t *siginfo, void *ptr)
* This also tests UFFD_FEATURE_EVENT_FORK event along with the signal
* feature. Using monitor thread, verify no userfault events are generated.
*/
-static int faulting_process(int signal_test)
+static int faulting_process(int signal_test, bool wp)
{
unsigned long nr;
unsigned long long count;
@@ -344,7 +345,7 @@ static int faulting_process(int signal_test)
if (steps == 1) {
/* This is a MISSING request */
steps++;
- if (copy_page(uffd, offset))
+ if (copy_page(uffd, offset, wp))
signalled++;
} else {
/* This is a WP request */
@@ -508,6 +509,7 @@ static int userfaultfd_events_test(void)
true, test_uffdio_wp, false))
err("register failure");
+ stats.apply_wp = test_uffdio_wp;
if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
err("uffd_poll_thread create");
@@ -516,7 +518,7 @@ static int userfaultfd_events_test(void)
err("fork");
if (!pid)
- exit(faulting_process(0));
+ exit(faulting_process(0, test_uffdio_wp));
waitpid(pid, &err, 0);
if (err)
@@ -552,11 +554,12 @@ static int userfaultfd_sig_test(void)
true, test_uffdio_wp, false))
err("register failure");
- if (faulting_process(1))
+ if (faulting_process(1, test_uffdio_wp))
err("faulting process failed");
uffd_test_ops->release_pages(area_dst);
+ stats.apply_wp = test_uffdio_wp;
if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
err("uffd_poll_thread create");
@@ -565,7 +568,7 @@ static int userfaultfd_sig_test(void)
err("fork");
if (!pid)
- exit(faulting_process(2));
+ exit(faulting_process(2, test_uffdio_wp));
waitpid(pid, &err, 0);
if (err)
@@ -629,6 +632,7 @@ static int userfaultfd_minor_test(void)
page_size);
}
+ stats.apply_wp = test_uffdio_wp;
if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
err("uffd_poll_thread create");
--
2.39.1
This moves the minor test to the new unit test.
Rewrite the content check with char* opeartions to avoid fiddling with
my_bcmp().
Drop global vars test_uffdio_minor and test_collapse, just assume test them
always in common code for now.
OTOH make this single test into five tests:
- minor test on [shmem, hugetlb] with wp=false
- minor test on [shmem, hugetlb] with wp=true
- minor test + collapse on shmem only
One thing to mention that we used to test COLLAPSE+WP but that doesn't
sound right at all. It's possible it's silently broken but unnoticed
because COLLAPSE is not part of the default test suite.
Make the MADV_COLLAPSE test fail-able (by skip it when failing), because
it's not guaranteed to success anyway.
Drop a bunch of useless code after the move, because the unit test always
use aligned num of pages and has nothing to do with n_cpus.
Cc: Zach O'Keefe <[email protected]>
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-common.c | 25 ++--
tools/testing/selftests/mm/uffd-common.h | 4 +-
tools/testing/selftests/mm/uffd-stress.c | 131 +------------------
tools/testing/selftests/mm/uffd-unit-tests.c | 120 +++++++++++++++++
4 files changed, 135 insertions(+), 145 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index ae6b61144b53..95ad619d0df4 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -13,8 +13,8 @@ volatile bool test_uffdio_copy_eexist = true;
unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
int uffd = -1, uffd_flags, finished, *pipefd, test_type;
-bool map_shared, test_collapse, test_dev_userfaultfd;
-bool test_uffdio_wp = true, test_uffdio_minor = false;
+bool map_shared, test_dev_userfaultfd;
+bool test_uffdio_wp = true;
unsigned long long *count_verify;
uffd_test_ops_t *uffd_test_ops;
@@ -126,28 +126,27 @@ static int shmem_allocate_area(void **alloc_area, bool is_src)
char *p = NULL, *p_alias = NULL;
int mem_fd = uffd_mem_fd_create(bytes * 2, false);
- if (test_collapse) {
- p = BASE_PMD_ADDR;
- if (!is_src)
- /* src map + alias + interleaved hpages */
- p += 2 * (bytes + hpage_size);
- p_alias = p;
- p_alias += bytes;
- p_alias += hpage_size; /* Prevent src/dst VMA merge */
- }
+ /* TODO: clean this up. Use a static addr is ugly */
+ p = BASE_PMD_ADDR;
+ if (!is_src)
+ /* src map + alias + interleaved hpages */
+ p += 2 * (bytes + hpage_size);
+ p_alias = p;
+ p_alias += bytes;
+ p_alias += hpage_size; /* Prevent src/dst VMA merge */
*alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
mem_fd, offset);
if (*alloc_area == MAP_FAILED)
err("mmap of memfd failed");
- if (test_collapse && *alloc_area != p)
+ if (*alloc_area != p)
err("mmap of memfd failed at %p", p);
area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
mem_fd, offset);
if (area_alias == MAP_FAILED)
err("mmap of memfd alias failed");
- if (test_collapse && area_alias != p_alias)
+ if (area_alias != p_alias)
err("mmap of anonymous memory failed at %p", p_alias);
if (is_src)
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 51ec75f6d0c1..16d32ddf8412 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -90,8 +90,8 @@ typedef struct uffd_test_ops uffd_test_ops_t;
extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
extern int uffd, uffd_flags, finished, *pipefd, test_type;
-extern bool map_shared, test_collapse, test_dev_userfaultfd;
-extern bool test_uffdio_wp, test_uffdio_minor;
+extern bool map_shared, test_dev_userfaultfd;
+extern bool test_uffdio_wp;
extern unsigned long long *count_verify;
extern volatile bool test_uffdio_copy_eexist;
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 50738a993afc..49fa61e5c54a 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -52,8 +52,6 @@ pthread_attr_t attr;
#define swap(a, b) \
do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
-#define factor_of_2(x) ((x) ^ ((x) & ((x) - 1)))
-
const char *examples =
"# Run anonymous memory test on 100MiB region with 99999 bounces:\n"
"./userfaultfd anon 100 99999\n\n"
@@ -79,8 +77,6 @@ static void usage(void)
"Supported mods:\n");
fprintf(stderr, "\tsyscall - Use userfaultfd(2) (default)\n");
fprintf(stderr, "\tdev - Use /dev/userfaultfd instead of userfaultfd(2)\n");
- fprintf(stderr, "\tcollapse - Test MADV_COLLAPSE of UFFDIO_REGISTER_MODE_MINOR\n"
- "memory\n");
fprintf(stderr, "\nExample test mod usage:\n");
fprintf(stderr, "# Run anonymous memory test with /dev/userfaultfd:\n");
fprintf(stderr, "./userfaultfd anon:dev 100 99999\n\n");
@@ -585,92 +581,6 @@ static int userfaultfd_sig_test(void)
return userfaults != 0;
}
-void check_memory_contents(char *p)
-{
- unsigned long i;
- uint8_t expected_byte;
- void *expected_page;
-
- if (posix_memalign(&expected_page, page_size, page_size))
- err("out of memory");
-
- for (i = 0; i < nr_pages; ++i) {
- expected_byte = ~((uint8_t)(i % ((uint8_t)-1)));
- memset(expected_page, expected_byte, page_size);
- if (my_bcmp(expected_page, p + (i * page_size), page_size))
- err("unexpected page contents after minor fault");
- }
-
- free(expected_page);
-}
-
-static int userfaultfd_minor_test(void)
-{
- unsigned long p;
- pthread_t uffd_mon;
- char c;
- struct uffd_stats stats = { 0 };
-
- if (!test_uffdio_minor)
- return 0;
-
- printf("testing minor faults: ");
- fflush(stdout);
-
- uffd_test_ctx_init(uffd_minor_feature());
-
- if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
- false, test_uffdio_wp, true))
- err("register failure");
-
- /*
- * After registering with UFFD, populate the non-UFFD-registered side of
- * the shared mapping. This should *not* trigger any UFFD minor faults.
- */
- for (p = 0; p < nr_pages; ++p) {
- memset(area_dst + (p * page_size), p % ((uint8_t)-1),
- page_size);
- }
-
- stats.apply_wp = test_uffdio_wp;
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
- err("uffd_poll_thread create");
-
- /*
- * Read each of the pages back using the UFFD-registered mapping. We
- * expect that the first time we touch a page, it will result in a minor
- * fault. uffd_poll_thread will resolve the fault by bit-flipping the
- * page's contents, and then issuing a CONTINUE ioctl.
- */
- check_memory_contents(area_dst_alias);
-
- if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
- err("pipe write");
- if (pthread_join(uffd_mon, NULL))
- return 1;
-
- uffd_stats_report(&stats, 1);
-
- if (test_collapse) {
- printf("testing collapse of uffd memory into PMD-mapped THPs:");
- if (madvise(area_dst_alias, nr_pages * page_size,
- MADV_COLLAPSE))
- err("madvise(MADV_COLLAPSE)");
-
- uffd_test_ops->check_pmd_mapping(area_dst,
- nr_pages * page_size /
- read_pmd_pagesize());
- /*
- * This won't cause uffd-fault - it purely just makes sure there
- * was no corruption.
- */
- check_memory_contents(area_dst_alias);
- printf(" done.\n");
- }
-
- return stats.missing_faults != 0 || stats.minor_faults != nr_pages;
-}
-
static int userfaultfd_stress(void)
{
void *area;
@@ -783,7 +693,7 @@ static int userfaultfd_stress(void)
}
return userfaultfd_zeropage_test() || userfaultfd_sig_test()
- || userfaultfd_events_test() || userfaultfd_minor_test();
+ || userfaultfd_events_test();
}
static void set_test_type(const char *type)
@@ -798,13 +708,10 @@ static void set_test_type(const char *type)
map_shared = true;
test_type = TEST_HUGETLB;
uffd_test_ops = &hugetlb_uffd_test_ops;
- /* Minor faults require shared hugetlb; only enable here. */
- test_uffdio_minor = true;
} else if (!strcmp(type, "shmem")) {
map_shared = true;
test_type = TEST_SHMEM;
uffd_test_ops = &shmem_uffd_test_ops;
- test_uffdio_minor = true;
}
}
@@ -822,8 +729,6 @@ static void parse_test_type_arg(const char *raw_type)
test_dev_userfaultfd = true;
else if (!strcmp(token, "syscall"))
test_dev_userfaultfd = false;
- else if (!strcmp(token, "collapse"))
- test_collapse = true;
else
err("unrecognized test mod '%s'", token);
}
@@ -831,9 +736,6 @@ static void parse_test_type_arg(const char *raw_type)
if (!test_type)
err("failed to parse test type argument: '%s'", raw_type);
- if (test_collapse && test_type != TEST_SHMEM)
- err("Unsupported test: %s", raw_type);
-
if (test_type == TEST_HUGETLB)
page_size = default_huge_page_size();
else
@@ -855,8 +757,6 @@ static void parse_test_type_arg(const char *raw_type)
test_uffdio_wp = test_uffdio_wp &&
(features & UFFD_FEATURE_PAGEFAULT_FLAG_WP);
- test_uffdio_minor = test_uffdio_minor &&
- (features & uffd_minor_feature());
close(uffd);
uffd = -1;
@@ -873,7 +773,6 @@ static void sigalrm(int sig)
int main(int argc, char **argv)
{
size_t bytes;
- size_t hpage_size = read_pmd_pagesize();
if (argc < 4)
usage();
@@ -885,36 +784,8 @@ int main(int argc, char **argv)
parse_test_type_arg(argv[1]);
bytes = atol(argv[2]) * 1024 * 1024;
- if (test_collapse && bytes & (hpage_size - 1))
- err("MiB must be multiple of %lu if :collapse mod set",
- hpage_size >> 20);
-
nr_cpus = sysconf(_SC_NPROCESSORS_ONLN);
- if (test_collapse) {
- /* nr_cpus must divide (bytes / page_size), otherwise,
- * area allocations of (nr_pages * paze_size) won't be a
- * multiple of hpage_size, even if bytes is a multiple of
- * hpage_size.
- *
- * This means that nr_cpus must divide (N * (2 << (H-P))
- * where:
- * bytes = hpage_size * N
- * hpage_size = 2 << H
- * page_size = 2 << P
- *
- * And we want to chose nr_cpus to be the largest value
- * satisfying this constraint, not larger than the number
- * of online CPUs. Unfortunately, prime factorization of
- * N and nr_cpus may be arbitrary, so have to search for it.
- * Instead, just use the highest power of 2 dividing both
- * nr_cpus and (bytes / page_size).
- */
- int x = factor_of_2(nr_cpus);
- int y = factor_of_2(bytes / page_size);
-
- nr_cpus = x < y ? x : y;
- }
nr_pages_per_cpu = bytes / page_size / nr_cpus;
if (!nr_pages_per_cpu) {
_err("invalid MiB");
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index bcd67bd4ec90..ecb8ba658736 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -332,6 +332,103 @@ static void uffd_pagemap_test(void)
uffd_test_pass();
}
+static void check_memory_contents(char *p)
+{
+ unsigned long i, j;
+ uint8_t expected_byte;
+
+ for (i = 0; i < nr_pages; ++i) {
+ expected_byte = ~((uint8_t)(i % ((uint8_t)-1)));
+ for (j = 0; j < page_size; j++) {
+ uint8_t v = *(uint8_t *)(p + (i * page_size) + j);
+ if (v != expected_byte)
+ err("unexpected page contents");
+ }
+ }
+}
+
+static void uffd_minor_test_common(bool test_collapse, bool test_wp)
+{
+ unsigned long p;
+ pthread_t uffd_mon;
+ char c;
+ struct uffd_stats stats = { 0 };
+
+ /*
+ * NOTE: MADV_COLLAPSE is not yet compatible with WP, so testing
+ * both do not make much sense.
+ */
+ assert(!(test_collapse && test_wp));
+
+ if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
+ /* NOTE! MADV_COLLAPSE may not work with uffd-wp */
+ false, test_wp, true))
+ err("register failure");
+
+ /*
+ * After registering with UFFD, populate the non-UFFD-registered side of
+ * the shared mapping. This should *not* trigger any UFFD minor faults.
+ */
+ for (p = 0; p < nr_pages; ++p)
+ memset(area_dst + (p * page_size), p % ((uint8_t)-1),
+ page_size);
+
+ stats.apply_wp = test_wp;
+ if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &stats))
+ err("uffd_poll_thread create");
+
+ /*
+ * Read each of the pages back using the UFFD-registered mapping. We
+ * expect that the first time we touch a page, it will result in a minor
+ * fault. uffd_poll_thread will resolve the fault by bit-flipping the
+ * page's contents, and then issuing a CONTINUE ioctl.
+ */
+ check_memory_contents(area_dst_alias);
+
+ if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
+ err("pipe write");
+ if (pthread_join(uffd_mon, NULL))
+ err("join() failed");
+
+ if (test_collapse) {
+ if (madvise(area_dst_alias, nr_pages * page_size,
+ MADV_COLLAPSE)) {
+ /* It's fine to fail for this one... */
+ uffd_test_skip("MADV_COLLAPSE failed");
+ return;
+ }
+
+ uffd_test_ops->check_pmd_mapping(area_dst,
+ nr_pages * page_size /
+ read_pmd_pagesize());
+ /*
+ * This won't cause uffd-fault - it purely just makes sure there
+ * was no corruption.
+ */
+ check_memory_contents(area_dst_alias);
+ }
+
+ if (stats.missing_faults != 0 || stats.minor_faults != nr_pages)
+ uffd_test_fail("stats check error");
+ else
+ uffd_test_pass();
+}
+
+void uffd_minor_test(void)
+{
+ uffd_minor_test_common(false, false);
+}
+
+void uffd_minor_wp_test(void)
+{
+ uffd_minor_test_common(false, true);
+}
+
+void uffd_minor_collapse_test(void)
+{
+ uffd_minor_test_common(true, false);
+}
+
uffd_test_case_t uffd_tests[] = {
{
.name = "pagemap",
@@ -346,6 +443,29 @@ uffd_test_case_t uffd_tests[] = {
.uffd_feature_required =
UFFD_FEATURE_PAGEFAULT_FLAG_WP | UFFD_FEATURE_WP_UNPOPULATED,
},
+ {
+ .name = "minor",
+ .uffd_fn = uffd_minor_test,
+ .mem_targets = MEM_SHMEM | MEM_HUGETLB,
+ .uffd_feature_required =
+ UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM,
+ },
+ {
+ .name = "minor-wp",
+ .uffd_fn = uffd_minor_wp_test,
+ .mem_targets = MEM_SHMEM | MEM_HUGETLB,
+ .uffd_feature_required =
+ UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM |
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ },
+ {
+ .name = "minor-collapse",
+ .uffd_fn = uffd_minor_collapse_test,
+ /* MADV_COLLAPSE only works with shmem */
+ .mem_targets = MEM_SHMEM,
+ /* We can't test MADV_COLLAPSE, so try our luck */
+ .uffd_feature_required = UFFD_FEATURE_MINOR_SHMEM,
+ },
};
int main(int argc, char *argv[])
--
2.39.1
With the new uffd unit test covering the /dev/userfaultfd path and syscall
path of uffd initializations, we can safely drop the devnode test in the
old stress test.
One thing is to avoid duplication of running the stress test twice which is
an overkill to only test the /dev/ interface in run_vmtests.sh.
The other benefit is now all uffd tests (that uses userfaultfd_open) can
run automatically as long as any type of interface is enabled (either
syscall or dev), so it's more likely to succeed rather than fail due to
unprivilege.
With this patch lands, we can drop all the "mem_type:XXX" handlings too.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/run_vmtests.sh | 15 +++++--------
tools/testing/selftests/mm/uffd-common.c | 7 ++----
tools/testing/selftests/mm/uffd-common.h | 2 +-
tools/testing/selftests/mm/uffd-stress.c | 27 ++---------------------
4 files changed, 11 insertions(+), 40 deletions(-)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index af7bbc74cd83..845ce8a48204 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -197,16 +197,13 @@ CATEGORY="gup_test" run_test ./gup_test -a
CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000
CATEGORY="userfaultfd" run_test ./uffd-unit-tests
-uffd_mods=("" ":dev")
uffd_stress_bin=./uffd-stress
-for mod in "${uffd_mods[@]}"; do
- CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16
- # Hugetlb tests require source and destination huge pages. Pass in half
- # the size ($half_ufd_size_MB), which is used for *each*.
- CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32
- CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32
- CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16
-done
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16
+# Hugetlb tests require source and destination huge pages. Pass in half
+# the size ($half_ufd_size_MB), which is used for *each*.
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared "$half_ufd_size_MB" 32
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16
#cleanup
echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 964fa57b710a..f14bf0941077 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -13,7 +13,7 @@ volatile bool test_uffdio_copy_eexist = true;
unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
int uffd = -1, uffd_flags, finished, *pipefd, test_type;
-bool map_shared, test_dev_userfaultfd;
+bool map_shared;
bool test_uffdio_wp = true;
unsigned long long *count_verify;
uffd_test_ops_t *uffd_test_ops;
@@ -229,10 +229,7 @@ int userfaultfd_open(uint64_t *features)
{
struct uffdio_api uffdio_api;
- if (test_dev_userfaultfd)
- uffd = uffd_open_dev(UFFD_FLAGS);
- else
- uffd = uffd_open_sys(UFFD_FLAGS);
+ uffd = uffd_open(UFFD_FLAGS);
if (uffd < 0)
return -1;
uffd_flags = fcntl(uffd, F_GETFD, NULL);
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 07d00e9809dc..f2e2fbb37cda 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -90,7 +90,7 @@ typedef struct uffd_test_ops uffd_test_ops_t;
extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
extern int uffd, uffd_flags, finished, *pipefd, test_type;
-extern bool map_shared, test_dev_userfaultfd;
+extern bool map_shared;
extern bool test_uffdio_wp;
extern unsigned long long *count_verify;
extern volatile bool test_uffdio_copy_eexist;
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 8f41bef2fbda..7461021dd4af 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -55,8 +55,6 @@ pthread_attr_t attr;
const char *examples =
"# Run anonymous memory test on 100MiB region with 99999 bounces:\n"
"./userfaultfd anon 100 99999\n\n"
- "# Run the same anonymous memory test, but using /dev/userfaultfd:\n"
- "./userfaultfd anon:dev 100 99999\n\n"
"# Run share memory test on 1GiB region with 99 bounces:\n"
"./userfaultfd shmem 1000 99\n\n"
"# Run hugetlb memory test on 256MiB region with 50 bounces:\n"
@@ -69,18 +67,9 @@ const char *examples =
static void usage(void)
{
- fprintf(stderr, "\nUsage: ./userfaultfd <test type> <MiB> <bounces> "
- "[hugetlbfs_file]\n\n");
+ fprintf(stderr, "\nUsage: ./userfaultfd <test type> <MiB> <bounces>\n\n");
fprintf(stderr, "Supported <test type>: anon, hugetlb, "
"hugetlb_shared, shmem\n\n");
- fprintf(stderr, "'Test mods' can be joined to the test type string with a ':'. "
- "Supported mods:\n");
- fprintf(stderr, "\tsyscall - Use userfaultfd(2) (default)\n");
- fprintf(stderr, "\tdev - Use /dev/userfaultfd instead of userfaultfd(2)\n");
- fprintf(stderr, "\nExample test mod usage:\n");
- fprintf(stderr, "# Run anonymous memory test with /dev/userfaultfd:\n");
- fprintf(stderr, "./userfaultfd anon:dev 100 99999\n\n");
-
fprintf(stderr, "Examples:\n\n");
fprintf(stderr, "%s", examples);
exit(1);
@@ -401,21 +390,9 @@ static void set_test_type(const char *type)
static void parse_test_type_arg(const char *raw_type)
{
- char *buf = strdup(raw_type);
uint64_t features = UFFD_API_FEATURES;
- while (buf) {
- const char *token = strsep(&buf, ":");
-
- if (!test_type)
- set_test_type(token);
- else if (!strcmp(token, "dev"))
- test_dev_userfaultfd = true;
- else if (!strcmp(token, "syscall"))
- test_dev_userfaultfd = false;
- else
- err("unrecognized test mod '%s'", token);
- }
+ set_test_type(raw_type);
if (!test_type)
err("failed to parse test type argument: '%s'", raw_type);
--
2.39.1
The userfaultfd stress test never tested private shmem, which I think was
overlooked long due. Add it so it matches with uffd unit test and it'll
cover all memory supported with the three memory types.
Meanwhile, rename the memory types a bit. Considering shared mem is the
major use case for both shmem / hugetlbfs, changing from:
anon, hugetlb, hugetlb_shared, shmem
To (with shmem-private added):
anon, hugetlb, hugetlb-private, shmem, shmem-private
Add the shmem-private to run_vmtests.sh too.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/run_vmtests.sh | 3 ++-
tools/testing/selftests/mm/uffd-stress.c | 11 +++++++----
2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index 845ce8a48204..b8ab75bc731c 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -202,8 +202,9 @@ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16
# Hugetlb tests require source and destination huge pages. Pass in half
# the size ($half_ufd_size_MB), which is used for *each*.
CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32
-CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared "$half_ufd_size_MB" 32
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private "$half_ufd_size_MB" 32
CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem-private 20 16
#cleanup
echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 7461021dd4af..6172ebfe732e 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -59,8 +59,8 @@ const char *examples =
"./userfaultfd shmem 1000 99\n\n"
"# Run hugetlb memory test on 256MiB region with 50 bounces:\n"
"./userfaultfd hugetlb 256 50\n\n"
- "# Run the same hugetlb test but using shared file:\n"
- "./userfaultfd hugetlb_shared 256 50\n\n"
+ "# Run the same hugetlb test but using private file:\n"
+ "./userfaultfd hugetlb-private 256 50\n\n"
"# 10MiB-~6GiB 999 bounces anonymous test, "
"continue forever unless an error triggers\n"
"while ./userfaultfd anon $[RANDOM % 6000 + 10] 999; do true; done\n\n";
@@ -69,7 +69,7 @@ static void usage(void)
{
fprintf(stderr, "\nUsage: ./userfaultfd <test type> <MiB> <bounces>\n\n");
fprintf(stderr, "Supported <test type>: anon, hugetlb, "
- "hugetlb_shared, shmem\n\n");
+ "hugetlb-private, shmem, shmem-private\n\n");
fprintf(stderr, "Examples:\n\n");
fprintf(stderr, "%s", examples);
exit(1);
@@ -377,14 +377,17 @@ static void set_test_type(const char *type)
} else if (!strcmp(type, "hugetlb")) {
test_type = TEST_HUGETLB;
uffd_test_ops = &hugetlb_uffd_test_ops;
- } else if (!strcmp(type, "hugetlb_shared")) {
map_shared = true;
+ } else if (!strcmp(type, "hugetlb-private")) {
test_type = TEST_HUGETLB;
uffd_test_ops = &hugetlb_uffd_test_ops;
} else if (!strcmp(type, "shmem")) {
map_shared = true;
test_type = TEST_SHMEM;
uffd_test_ops = &shmem_uffd_test_ops;
+ } else if (!strcmp(type, "shmem-private")) {
+ test_type = TEST_SHMEM;
+ uffd_test_ops = &shmem_uffd_test_ops;
}
}
--
2.39.1
Move the two tests into the unit test, and convert it into 20 standalone
tests:
- events test on all 5 mem types, with wp on/off
- signal test on all 5 mem types, with wp on/off
Testing sigbus on anon... done
Testing sigbus on shmem... done
Testing sigbus on shmem-private... done
Testing sigbus on hugetlb... done
Testing sigbus on hugetlb-private... done
Testing sigbus-wp on anon... done
Testing sigbus-wp on shmem... done
Testing sigbus-wp on shmem-private... done
Testing sigbus-wp on hugetlb... done
Testing sigbus-wp on hugetlb-private... done
Testing events on anon... done
Testing events on shmem... done
Testing events on shmem-private... done
Testing events on hugetlb... done
Testing events on hugetlb-private... done
Testing events-wp on anon... done
Testing events-wp on shmem... done
Testing events-wp on shmem-private... done
Testing events-wp on hugetlb... done
Testing events-wp on hugetlb-private... done
It'll also remove a lot of global references along the way,
e.g. test_uffdio_wp will be replaced with the wp value passed over.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-stress.c | 227 +---------------
tools/testing/selftests/mm/uffd-unit-tests.c | 263 +++++++++++++++++++
2 files changed, 264 insertions(+), 226 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 49fa61e5c54a..f3046ae13a90 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -274,133 +274,6 @@ static int stress(struct uffd_stats *uffd_stats)
return 0;
}
-sigjmp_buf jbuf, *sigbuf;
-
-static void sighndl(int sig, siginfo_t *siginfo, void *ptr)
-{
- if (sig == SIGBUS) {
- if (sigbuf)
- siglongjmp(*sigbuf, 1);
- abort();
- }
-}
-
-/*
- * For non-cooperative userfaultfd test we fork() a process that will
- * generate pagefaults, will mremap the area monitored by the
- * userfaultfd and at last this process will release the monitored
- * area.
- * For the anonymous and shared memory the area is divided into two
- * parts, the first part is accessed before mremap, and the second
- * part is accessed after mremap. Since hugetlbfs does not support
- * mremap, the entire monitored area is accessed in a single pass for
- * HUGETLB_TEST.
- * The release of the pages currently generates event for shmem and
- * anonymous memory (UFFD_EVENT_REMOVE), hence it is not checked
- * for hugetlb.
- * For signal test(UFFD_FEATURE_SIGBUS), signal_test = 1, we register
- * monitored area, generate pagefaults and test that signal is delivered.
- * Use UFFDIO_COPY to allocate missing page and retry. For signal_test = 2
- * test robustness use case - we release monitored area, fork a process
- * that will generate pagefaults and verify signal is generated.
- * This also tests UFFD_FEATURE_EVENT_FORK event along with the signal
- * feature. Using monitor thread, verify no userfault events are generated.
- */
-static int faulting_process(int signal_test, bool wp)
-{
- unsigned long nr;
- unsigned long long count;
- unsigned long split_nr_pages;
- unsigned long lastnr;
- struct sigaction act;
- volatile unsigned long signalled = 0;
-
- split_nr_pages = (nr_pages + 1) / 2;
-
- if (signal_test) {
- sigbuf = &jbuf;
- memset(&act, 0, sizeof(act));
- act.sa_sigaction = sighndl;
- act.sa_flags = SA_SIGINFO;
- if (sigaction(SIGBUS, &act, 0))
- err("sigaction");
- lastnr = (unsigned long)-1;
- }
-
- for (nr = 0; nr < split_nr_pages; nr++) {
- volatile int steps = 1;
- unsigned long offset = nr * page_size;
-
- if (signal_test) {
- if (sigsetjmp(*sigbuf, 1) != 0) {
- if (steps == 1 && nr == lastnr)
- err("Signal repeated");
-
- lastnr = nr;
- if (signal_test == 1) {
- if (steps == 1) {
- /* This is a MISSING request */
- steps++;
- if (copy_page(uffd, offset, wp))
- signalled++;
- } else {
- /* This is a WP request */
- assert(steps == 2);
- wp_range(uffd,
- (__u64)area_dst +
- offset,
- page_size, false);
- }
- } else {
- signalled++;
- continue;
- }
- }
- }
-
- count = *area_count(area_dst, nr);
- if (count != count_verify[nr])
- err("nr %lu memory corruption %llu %llu\n",
- nr, count, count_verify[nr]);
- /*
- * Trigger write protection if there is by writing
- * the same value back.
- */
- *area_count(area_dst, nr) = count;
- }
-
- if (signal_test)
- return signalled != split_nr_pages;
-
- area_dst = mremap(area_dst, nr_pages * page_size, nr_pages * page_size,
- MREMAP_MAYMOVE | MREMAP_FIXED, area_src);
- if (area_dst == MAP_FAILED)
- err("mremap");
- /* Reset area_src since we just clobbered it */
- area_src = NULL;
-
- for (; nr < nr_pages; nr++) {
- count = *area_count(area_dst, nr);
- if (count != count_verify[nr]) {
- err("nr %lu memory corruption %llu %llu\n",
- nr, count, count_verify[nr]);
- }
- /*
- * Trigger write protection if there is by writing
- * the same value back.
- */
- *area_count(area_dst, nr) = count;
- }
-
- uffd_test_ops->release_pages(area_dst);
-
- for (nr = 0; nr < nr_pages; nr++)
- if (my_bcmp(area_dst + nr * page_size, zeropage, page_size))
- err("nr %lu is not zero", nr);
-
- return 0;
-}
-
static void retry_uffdio_zeropage(int ufd,
struct uffdio_zeropage *uffdio_zeropage,
unsigned long offset)
@@ -484,103 +357,6 @@ static int userfaultfd_zeropage_test(void)
return 0;
}
-static int userfaultfd_events_test(void)
-{
- pthread_t uffd_mon;
- int err, features;
- pid_t pid;
- char c;
- struct uffd_stats stats = { 0 };
-
- printf("testing events (fork, remap, remove): ");
- fflush(stdout);
-
- features = UFFD_FEATURE_EVENT_FORK | UFFD_FEATURE_EVENT_REMAP |
- UFFD_FEATURE_EVENT_REMOVE;
- uffd_test_ctx_init(features);
-
- fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
-
- if (uffd_register(uffd, area_dst, nr_pages * page_size,
- true, test_uffdio_wp, false))
- err("register failure");
-
- stats.apply_wp = test_uffdio_wp;
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
- err("uffd_poll_thread create");
-
- pid = fork();
- if (pid < 0)
- err("fork");
-
- if (!pid)
- exit(faulting_process(0, test_uffdio_wp));
-
- waitpid(pid, &err, 0);
- if (err)
- err("faulting process failed");
- if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
- err("pipe write");
- if (pthread_join(uffd_mon, NULL))
- return 1;
-
- uffd_stats_report(&stats, 1);
-
- return stats.missing_faults != nr_pages;
-}
-
-static int userfaultfd_sig_test(void)
-{
- unsigned long userfaults;
- pthread_t uffd_mon;
- int err, features;
- pid_t pid;
- char c;
- struct uffd_stats stats = { 0 };
-
- printf("testing signal delivery: ");
- fflush(stdout);
-
- features = UFFD_FEATURE_EVENT_FORK|UFFD_FEATURE_SIGBUS;
- uffd_test_ctx_init(features);
-
- fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
-
- if (uffd_register(uffd, area_dst, nr_pages * page_size,
- true, test_uffdio_wp, false))
- err("register failure");
-
- if (faulting_process(1, test_uffdio_wp))
- err("faulting process failed");
-
- uffd_test_ops->release_pages(area_dst);
-
- stats.apply_wp = test_uffdio_wp;
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
- err("uffd_poll_thread create");
-
- pid = fork();
- if (pid < 0)
- err("fork");
-
- if (!pid)
- exit(faulting_process(2, test_uffdio_wp));
-
- waitpid(pid, &err, 0);
- if (err)
- err("faulting process failed");
- if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
- err("pipe write");
- if (pthread_join(uffd_mon, (void **)&userfaults))
- return 1;
-
- printf("done.\n");
- if (userfaults)
- err("Signal test failed, userfaults: %ld", userfaults);
-
- return userfaults != 0;
-}
-
static int userfaultfd_stress(void)
{
void *area;
@@ -692,8 +468,7 @@ static int userfaultfd_stress(void)
uffd_stats_report(uffd_stats, nr_cpus);
}
- return userfaultfd_zeropage_test() || userfaultfd_sig_test()
- || userfaultfd_events_test();
+ return userfaultfd_zeropage_test();
}
static void set_test_type(const char *type)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index ecb8ba658736..ebf45cb0eca8 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -18,6 +18,9 @@
#define MEM_HUGETLB BIT_ULL(3)
#define MEM_HUGETLB_PRIVATE BIT_ULL(4)
+#define MEM_ALL (MEM_ANON | MEM_SHMEM | MEM_SHMEM_PRIVATE | \
+ MEM_HUGETLB | MEM_HUGETLB_PRIVATE)
+
struct mem_type {
const char *name;
unsigned int mem_flag;
@@ -429,6 +432,237 @@ void uffd_minor_collapse_test(void)
uffd_minor_test_common(true, false);
}
+static sigjmp_buf jbuf, *sigbuf;
+
+static void sighndl(int sig, siginfo_t *siginfo, void *ptr)
+{
+ if (sig == SIGBUS) {
+ if (sigbuf)
+ siglongjmp(*sigbuf, 1);
+ abort();
+ }
+}
+
+/*
+ * For non-cooperative userfaultfd test we fork() a process that will
+ * generate pagefaults, will mremap the area monitored by the
+ * userfaultfd and at last this process will release the monitored
+ * area.
+ * For the anonymous and shared memory the area is divided into two
+ * parts, the first part is accessed before mremap, and the second
+ * part is accessed after mremap. Since hugetlbfs does not support
+ * mremap, the entire monitored area is accessed in a single pass for
+ * HUGETLB_TEST.
+ * The release of the pages currently generates event for shmem and
+ * anonymous memory (UFFD_EVENT_REMOVE), hence it is not checked
+ * for hugetlb.
+ * For signal test(UFFD_FEATURE_SIGBUS), signal_test = 1, we register
+ * monitored area, generate pagefaults and test that signal is delivered.
+ * Use UFFDIO_COPY to allocate missing page and retry. For signal_test = 2
+ * test robustness use case - we release monitored area, fork a process
+ * that will generate pagefaults and verify signal is generated.
+ * This also tests UFFD_FEATURE_EVENT_FORK event along with the signal
+ * feature. Using monitor thread, verify no userfault events are generated.
+ */
+static int faulting_process(int signal_test, bool wp)
+{
+ unsigned long nr, i;
+ unsigned long long count;
+ unsigned long split_nr_pages;
+ unsigned long lastnr;
+ struct sigaction act;
+ volatile unsigned long signalled = 0;
+
+ split_nr_pages = (nr_pages + 1) / 2;
+
+ if (signal_test) {
+ sigbuf = &jbuf;
+ memset(&act, 0, sizeof(act));
+ act.sa_sigaction = sighndl;
+ act.sa_flags = SA_SIGINFO;
+ if (sigaction(SIGBUS, &act, 0))
+ err("sigaction");
+ lastnr = (unsigned long)-1;
+ }
+
+ for (nr = 0; nr < split_nr_pages; nr++) {
+ volatile int steps = 1;
+ unsigned long offset = nr * page_size;
+
+ if (signal_test) {
+ if (sigsetjmp(*sigbuf, 1) != 0) {
+ if (steps == 1 && nr == lastnr)
+ err("Signal repeated");
+
+ lastnr = nr;
+ if (signal_test == 1) {
+ if (steps == 1) {
+ /* This is a MISSING request */
+ steps++;
+ if (copy_page(uffd, offset, wp))
+ signalled++;
+ } else {
+ /* This is a WP request */
+ assert(steps == 2);
+ wp_range(uffd,
+ (__u64)area_dst +
+ offset,
+ page_size, false);
+ }
+ } else {
+ signalled++;
+ continue;
+ }
+ }
+ }
+
+ count = *area_count(area_dst, nr);
+ if (count != count_verify[nr])
+ err("nr %lu memory corruption %llu %llu\n",
+ nr, count, count_verify[nr]);
+ /*
+ * Trigger write protection if there is by writing
+ * the same value back.
+ */
+ *area_count(area_dst, nr) = count;
+ }
+
+ if (signal_test)
+ return signalled != split_nr_pages;
+
+ area_dst = mremap(area_dst, nr_pages * page_size, nr_pages * page_size,
+ MREMAP_MAYMOVE | MREMAP_FIXED, area_src);
+ if (area_dst == MAP_FAILED)
+ err("mremap");
+ /* Reset area_src since we just clobbered it */
+ area_src = NULL;
+
+ for (; nr < nr_pages; nr++) {
+ count = *area_count(area_dst, nr);
+ if (count != count_verify[nr]) {
+ err("nr %lu memory corruption %llu %llu\n",
+ nr, count, count_verify[nr]);
+ }
+ /*
+ * Trigger write protection if there is by writing
+ * the same value back.
+ */
+ *area_count(area_dst, nr) = count;
+ }
+
+ uffd_test_ops->release_pages(area_dst);
+
+ for (nr = 0; nr < nr_pages; nr++)
+ for (i = 0; i < page_size; i++)
+ if (*(area_dst + nr * page_size + i) != 0)
+ err("page %lu offset %lu is not zero", nr, i);
+
+ return 0;
+}
+
+static void uffd_sigbus_test_common(bool wp)
+{
+ unsigned long userfaults;
+ pthread_t uffd_mon;
+ pid_t pid;
+ int err;
+ char c;
+ struct uffd_stats stats = { 0 };
+
+ fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
+
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, wp, false))
+ err("register failure");
+
+ if (faulting_process(1, wp))
+ err("faulting process failed");
+
+ uffd_test_ops->release_pages(area_dst);
+
+ stats.apply_wp = wp;
+ if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &stats))
+ err("uffd_poll_thread create");
+
+ pid = fork();
+ if (pid < 0)
+ err("fork");
+
+ if (!pid)
+ exit(faulting_process(2, wp));
+
+ waitpid(pid, &err, 0);
+ if (err)
+ err("faulting process failed");
+ if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
+ err("pipe write");
+ if (pthread_join(uffd_mon, (void **)&userfaults))
+ err("pthread_join()");
+
+ if (userfaults)
+ uffd_test_fail("Signal test failed, userfaults: %ld", userfaults);
+ else
+ uffd_test_pass();
+}
+
+static void uffd_sigbus_test(void)
+{
+ uffd_sigbus_test_common(false);
+}
+
+static void uffd_sigbus_wp_test(void)
+{
+ uffd_sigbus_test_common(true);
+}
+
+static void uffd_events_test_common(bool wp)
+{
+ pthread_t uffd_mon;
+ pid_t pid;
+ int err;
+ char c;
+ struct uffd_stats stats = { 0 };
+
+ fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, wp, false))
+ err("register failure");
+
+ stats.apply_wp = wp;
+ if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &stats))
+ err("uffd_poll_thread create");
+
+ pid = fork();
+ if (pid < 0)
+ err("fork");
+
+ if (!pid)
+ exit(faulting_process(0, wp));
+
+ waitpid(pid, &err, 0);
+ if (err)
+ err("faulting process failed");
+ if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
+ err("pipe write");
+ if (pthread_join(uffd_mon, NULL))
+ err("pthread_join()");
+
+ if (stats.missing_faults != nr_pages)
+ uffd_test_fail("Fault counts wrong");
+ else
+ uffd_test_pass();
+}
+
+static void uffd_events_test(void)
+{
+ uffd_events_test_common(false);
+}
+
+static void uffd_events_wp_test(void)
+{
+ uffd_events_test_common(true);
+}
+
uffd_test_case_t uffd_tests[] = {
{
.name = "pagemap",
@@ -466,6 +700,35 @@ uffd_test_case_t uffd_tests[] = {
/* We can't test MADV_COLLAPSE, so try our luck */
.uffd_feature_required = UFFD_FEATURE_MINOR_SHMEM,
},
+ {
+ .name = "sigbus",
+ .uffd_fn = uffd_sigbus_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_SIGBUS |
+ UFFD_FEATURE_EVENT_FORK,
+ },
+ {
+ .name = "sigbus-wp",
+ .uffd_fn = uffd_sigbus_wp_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_SIGBUS |
+ UFFD_FEATURE_EVENT_FORK | UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ },
+ {
+ .name = "events",
+ .uffd_fn = uffd_events_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_EVENT_FORK |
+ UFFD_FEATURE_EVENT_REMAP | UFFD_FEATURE_EVENT_REMOVE,
+ },
+ {
+ .name = "events-wp",
+ .uffd_fn = uffd_events_wp_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_EVENT_FORK |
+ UFFD_FEATURE_EVENT_REMAP | UFFD_FEATURE_EVENT_REMOVE |
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ },
};
int main(int argc, char *argv[])
--
2.39.1
Simplifies it a bit along the way, e.g., drop the never used offset
field (which was always the 1st page so offset=0).
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-stress.c | 94 +-------------------
tools/testing/selftests/mm/uffd-unit-tests.c | 94 ++++++++++++++++++++
2 files changed, 95 insertions(+), 93 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index f3046ae13a90..a6f3609c1ad1 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -110,15 +110,6 @@ static inline uint64_t uffd_minor_feature(void)
return 0;
}
-static int my_bcmp(char *str1, char *str2, size_t n)
-{
- unsigned long i;
- for (i = 0; i < n; i++)
- if (str1[i] != str2[i])
- return 1;
- return 0;
-}
-
static void *locking_thread(void *arg)
{
unsigned long cpu = (unsigned long) arg;
@@ -274,89 +265,6 @@ static int stress(struct uffd_stats *uffd_stats)
return 0;
}
-static void retry_uffdio_zeropage(int ufd,
- struct uffdio_zeropage *uffdio_zeropage,
- unsigned long offset)
-{
- uffd_test_ops->alias_mapping(&uffdio_zeropage->range.start,
- uffdio_zeropage->range.len,
- offset);
- if (ioctl(ufd, UFFDIO_ZEROPAGE, uffdio_zeropage)) {
- if (uffdio_zeropage->zeropage != -EEXIST)
- err("UFFDIO_ZEROPAGE error: %"PRId64,
- (int64_t)uffdio_zeropage->zeropage);
- } else {
- err("UFFDIO_ZEROPAGE error: %"PRId64,
- (int64_t)uffdio_zeropage->zeropage);
- }
-}
-
-static int __uffdio_zeropage(int ufd, unsigned long offset)
-{
- struct uffdio_zeropage uffdio_zeropage;
- int ret;
- bool has_zeropage = !(test_type == TEST_HUGETLB);
- __s64 res;
-
- if (offset >= nr_pages * page_size)
- err("unexpected offset %lu", offset);
- uffdio_zeropage.range.start = (unsigned long) area_dst + offset;
- uffdio_zeropage.range.len = page_size;
- uffdio_zeropage.mode = 0;
- ret = ioctl(ufd, UFFDIO_ZEROPAGE, &uffdio_zeropage);
- res = uffdio_zeropage.zeropage;
- if (ret) {
- /* real retval in ufdio_zeropage.zeropage */
- if (has_zeropage)
- err("UFFDIO_ZEROPAGE error: %"PRId64, (int64_t)res);
- else if (res != -EINVAL)
- err("UFFDIO_ZEROPAGE not -EINVAL");
- } else if (has_zeropage) {
- if (res != page_size) {
- err("UFFDIO_ZEROPAGE unexpected size");
- } else {
- retry_uffdio_zeropage(ufd, &uffdio_zeropage,
- offset);
- return 1;
- }
- } else
- err("UFFDIO_ZEROPAGE succeeded");
-
- return 0;
-}
-
-static int uffdio_zeropage(int ufd, unsigned long offset)
-{
- return __uffdio_zeropage(ufd, offset);
-}
-
-/* exercise UFFDIO_ZEROPAGE */
-static int userfaultfd_zeropage_test(void)
-{
- printf("testing UFFDIO_ZEROPAGE: ");
- fflush(stdout);
-
- uffd_test_ctx_init(0);
-
- if (uffd_register(uffd, area_dst, nr_pages * page_size,
- true, test_uffdio_wp, false))
- err("register failure");
-
- if (area_dst_alias) {
- /* Needed this to test zeropage-retry on shared memory */
- if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
- true, test_uffdio_wp, false))
- err("register failure");
- }
-
- if (uffdio_zeropage(uffd, 0))
- if (my_bcmp(area_dst, zeropage, page_size))
- err("zeropage is not zero");
-
- printf("done.\n");
- return 0;
-}
-
static int userfaultfd_stress(void)
{
void *area;
@@ -468,7 +376,7 @@ static int userfaultfd_stress(void)
uffd_stats_report(uffd_stats, nr_cpus);
}
- return userfaultfd_zeropage_test();
+ return 0;
}
static void set_test_type(const char *type)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index ebf45cb0eca8..376dfa320b6f 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -663,7 +663,101 @@ static void uffd_events_wp_test(void)
uffd_events_test_common(true);
}
+static void retry_uffdio_zeropage(int ufd,
+ struct uffdio_zeropage *uffdio_zeropage)
+{
+ uffd_test_ops->alias_mapping(&uffdio_zeropage->range.start,
+ uffdio_zeropage->range.len,
+ 0);
+ if (ioctl(ufd, UFFDIO_ZEROPAGE, uffdio_zeropage)) {
+ if (uffdio_zeropage->zeropage != -EEXIST)
+ err("UFFDIO_ZEROPAGE error: %"PRId64,
+ (int64_t)uffdio_zeropage->zeropage);
+ } else {
+ err("UFFDIO_ZEROPAGE error: %"PRId64,
+ (int64_t)uffdio_zeropage->zeropage);
+ }
+}
+
+static bool do_uffdio_zeropage(int ufd, bool has_zeropage)
+{
+ struct uffdio_zeropage uffdio_zeropage = { 0 };
+ int ret;
+ __s64 res;
+
+ uffdio_zeropage.range.start = (unsigned long) area_dst;
+ uffdio_zeropage.range.len = page_size;
+ uffdio_zeropage.mode = 0;
+ ret = ioctl(ufd, UFFDIO_ZEROPAGE, &uffdio_zeropage);
+ res = uffdio_zeropage.zeropage;
+ if (ret) {
+ /* real retval in ufdio_zeropage.zeropage */
+ if (has_zeropage)
+ err("UFFDIO_ZEROPAGE error: %"PRId64, (int64_t)res);
+ else if (res != -EINVAL)
+ err("UFFDIO_ZEROPAGE not -EINVAL");
+ } else if (has_zeropage) {
+ if (res != page_size)
+ err("UFFDIO_ZEROPAGE unexpected size");
+ else
+ retry_uffdio_zeropage(ufd, &uffdio_zeropage);
+ return true;
+ } else
+ err("UFFDIO_ZEROPAGE succeeded");
+
+ return false;
+}
+
+/* exercise UFFDIO_ZEROPAGE */
+static void uffd_zeropage_test_common(bool has_zeropage)
+{
+ if (uffd_register(uffd, area_dst, page_size,
+ true, false, false))
+ err("register");
+
+ if (area_dst_alias)
+ if (uffd_register(uffd, area_dst_alias, page_size,
+ true, false, false))
+ err("register");
+
+ if (do_uffdio_zeropage(uffd, has_zeropage)) {
+ int i;
+
+ for (i = 0; i < page_size; i++)
+ if (area_dst[i] != 0)
+ err("data non-zero at offset %d\n", i);
+ }
+
+
+ if (uffd_unregister(uffd, area_dst, page_size * nr_pages))
+ err("unregister");
+
+ uffd_test_pass();
+}
+
+static void uffd_zeropage_test(void)
+{
+ uffd_zeropage_test_common(true);
+}
+
+static void uffd_zeropage_hugetlb_test(void)
+{
+ uffd_zeropage_test_common(false);
+}
+
uffd_test_case_t uffd_tests[] = {
+ {
+ .name = "zeropage",
+ .uffd_fn = uffd_zeropage_test,
+ .mem_targets = MEM_ANON | MEM_SHMEM | MEM_SHMEM_PRIVATE,
+ .uffd_feature_required = 0,
+ },
+ {
+ .name = "zeropage-hugetlb",
+ .uffd_fn = uffd_zeropage_hugetlb_test,
+ .mem_targets = MEM_HUGETLB | MEM_HUGETLB_PRIVATE,
+ .uffd_feature_required = 0,
+ },
{
.name = "pagemap",
.uffd_fn = uffd_pagemap_test,
--
2.39.1
Mostly to detect hugetlb allocation errors and skip hugetlb tests when
pages are not allocated.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-common.c | 32 +++++++++++++++++-------
tools/testing/selftests/mm/uffd-common.h | 4 +--
2 files changed, 25 insertions(+), 11 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 92b7e00efa8a..ae6b61144b53 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -44,10 +44,13 @@ static void anon_release_pages(char *rel_area)
err("madvise(MADV_DONTNEED) failed");
}
-static void anon_allocate_area(void **alloc_area, bool is_src)
+static int anon_allocate_area(void **alloc_area, bool is_src)
{
*alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+ if (*alloc_area == MAP_FAILED)
+ err("ENOMEM");
+ return 0;
}
static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
@@ -65,7 +68,7 @@ static void hugetlb_release_pages(char *rel_area)
}
}
-static void hugetlb_allocate_area(void **alloc_area, bool is_src)
+static int hugetlb_allocate_area(void **alloc_area, bool is_src)
{
off_t size = nr_pages * page_size;
off_t offset = is_src ? 0 : size;
@@ -77,14 +80,16 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
(map_shared ? MAP_SHARED : MAP_PRIVATE) |
(is_src ? 0 : MAP_NORESERVE),
mem_fd, offset);
- if (*alloc_area == MAP_FAILED)
- err("mmap of hugetlbfs file failed");
+ if (*alloc_area == MAP_FAILED) {
+ *alloc_area = NULL;
+ return -errno;
+ }
if (map_shared) {
area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
MAP_SHARED, mem_fd, offset);
if (area_alias == MAP_FAILED)
- err("mmap of hugetlb file alias failed");
+ return -errno;
}
if (is_src) {
@@ -96,6 +101,7 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
*alloc_area_alias = area_alias;
close(mem_fd);
+ return 0;
}
static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
@@ -112,7 +118,7 @@ static void shmem_release_pages(char *rel_area)
err("madvise(MADV_REMOVE) failed");
}
-static void shmem_allocate_area(void **alloc_area, bool is_src)
+static int shmem_allocate_area(void **alloc_area, bool is_src)
{
void *area_alias = NULL;
size_t bytes = nr_pages * page_size, hpage_size = read_pmd_pagesize();
@@ -150,6 +156,7 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
area_dst_alias = area_alias;
close(mem_fd);
+ return 0;
}
static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
@@ -282,14 +289,19 @@ static void uffd_test_ctx_clear(void)
munmap_area((void **)&area_remap);
}
-void uffd_test_ctx_init(uint64_t features)
+int uffd_test_ctx_init(uint64_t features)
{
unsigned long nr, cpu;
+ int ret;
uffd_test_ctx_clear();
- uffd_test_ops->allocate_area((void **)&area_src, true);
- uffd_test_ops->allocate_area((void **)&area_dst, false);
+ ret = uffd_test_ops->allocate_area((void **)&area_src, true);
+ if (ret)
+ return ret;
+ ret = uffd_test_ops->allocate_area((void **)&area_dst, false);
+ if (ret)
+ return ret;
userfaultfd_open(&features);
@@ -337,6 +349,8 @@ void uffd_test_ctx_init(uint64_t features)
for (cpu = 0; cpu < nr_cpus; cpu++)
if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
err("pipe");
+
+ return 0;
}
void wp_range(int ufd, __u64 start, __u64 len, bool wp)
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index f4bc73ce3b48..51ec75f6d0c1 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -80,7 +80,7 @@ struct uffd_stats {
};
struct uffd_test_ops {
- void (*allocate_area)(void **alloc_area, bool is_src);
+ int (*allocate_area)(void **alloc_area, bool is_src);
void (*release_pages)(char *rel_area);
void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
@@ -101,7 +101,7 @@ extern uffd_test_ops_t hugetlb_uffd_test_ops;
extern uffd_test_ops_t *uffd_test_ops;
void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
-void uffd_test_ctx_init(uint64_t features);
+int uffd_test_ctx_init(uint64_t features);
void userfaultfd_open(uint64_t *features);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
--
2.39.1
Userfaultfd minor+wp mode was very recently added. The test will fail on
the old kernels at ioctl(UFFDIO_CONTINUE) which is misterious.
Unfortunately there's no feature bit to detect for this support.
Add a hack to leverage WP_UNPOPULATED to detect whether that feature
existed, since WP_UNPOPULATED was merged right after minor+wp.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 376dfa320b6f..82fd3aaa06c2 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -784,7 +784,13 @@ uffd_test_case_t uffd_tests[] = {
.mem_targets = MEM_SHMEM | MEM_HUGETLB,
.uffd_feature_required =
UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM |
- UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP |
+ /*
+ * HACK: here we leveraged WP_UNPOPULATED to detect whether
+ * minor mode supports wr-protect. There's no feature flag
+ * for it so this is the best we can test against.
+ */
+ UFFD_FEATURE_WP_UNPOPULATED,
},
{
.name = "minor-collapse",
--
2.39.1
Allow skip a unit test properly due to no privilege (e.g. sigbus and events
tests).
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-common.c | 27 ++++++++++++--------
tools/testing/selftests/mm/uffd-common.h | 4 +--
tools/testing/selftests/mm/uffd-stress.c | 6 +++--
tools/testing/selftests/mm/uffd-unit-tests.c | 10 +++++---
4 files changed, 29 insertions(+), 18 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 95ad619d0df4..964fa57b710a 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -225,7 +225,7 @@ void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
printf("\n");
}
-void userfaultfd_open(uint64_t *features)
+int userfaultfd_open(uint64_t *features)
{
struct uffdio_api uffdio_api;
@@ -234,18 +234,19 @@ void userfaultfd_open(uint64_t *features)
else
uffd = uffd_open_sys(UFFD_FLAGS);
if (uffd < 0)
- err("uffd open failed (dev=%d)", test_dev_userfaultfd);
+ return -1;
uffd_flags = fcntl(uffd, F_GETFD, NULL);
uffdio_api.api = UFFD_API;
uffdio_api.features = *features;
if (ioctl(uffd, UFFDIO_API, &uffdio_api))
- err("UFFDIO_API failed.\nPlease make sure to "
- "run with either root or ptrace capability.");
+ /* Probably lack of CAP_PTRACE? */
+ return -1;
if (uffdio_api.api != UFFD_API)
err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api);
*features = uffdio_api.features;
+ return 0;
}
static inline void munmap_area(void **area)
@@ -288,7 +289,7 @@ static void uffd_test_ctx_clear(void)
munmap_area((void **)&area_remap);
}
-int uffd_test_ctx_init(uint64_t features)
+int uffd_test_ctx_init(uint64_t features, const char **errmsg)
{
unsigned long nr, cpu;
int ret;
@@ -296,13 +297,19 @@ int uffd_test_ctx_init(uint64_t features)
uffd_test_ctx_clear();
ret = uffd_test_ops->allocate_area((void **)&area_src, true);
- if (ret)
- return ret;
- ret = uffd_test_ops->allocate_area((void **)&area_dst, false);
- if (ret)
+ ret |= uffd_test_ops->allocate_area((void **)&area_dst, false);
+ if (ret) {
+ if (errmsg)
+ *errmsg = "memory allocation failed";
return ret;
+ }
- userfaultfd_open(&features);
+ ret = userfaultfd_open(&features);
+ if (ret) {
+ if (errmsg)
+ *errmsg = "possible lack of priviledge";
+ return ret;
+ }
count_verify = malloc(nr_pages * sizeof(unsigned long long));
if (!count_verify)
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 16d32ddf8412..07d00e9809dc 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -101,8 +101,8 @@ extern uffd_test_ops_t hugetlb_uffd_test_ops;
extern uffd_test_ops_t *uffd_test_ops;
void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
-int uffd_test_ctx_init(uint64_t features);
-void userfaultfd_open(uint64_t *features);
+int uffd_test_ctx_init(uint64_t features, const char **errmsg);
+int userfaultfd_open(uint64_t *features);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index a6f3609c1ad1..8f41bef2fbda 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -272,7 +272,8 @@ static int userfaultfd_stress(void)
struct uffd_stats uffd_stats[nr_cpus];
uint64_t mem_size = nr_pages * page_size;
- uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
+ if (uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED, NULL))
+ err("context init failed");
if (posix_memalign(&area, page_size, page_size))
err("out of memory");
@@ -436,7 +437,8 @@ static void parse_test_type_arg(const char *raw_type)
* feature.
*/
- userfaultfd_open(&features);
+ if (userfaultfd_open(&features))
+ err("Userfaultfd open failed");
test_uffdio_wp = test_uffdio_wp &&
(features & UFFD_FEATURE_PAGEFAULT_FLAG_WP);
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 82fd3aaa06c2..9e7f7c7f2982 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -175,7 +175,8 @@ static int test_uffd_api(bool use_dev)
* This function initializes the global variables. TODO: remove global
* vars and then remove this.
*/
-static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type)
+static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type,
+ const char **errmsg)
{
map_shared = mem_type->shared;
uffd_test_ops = mem_type->mem_ops;
@@ -189,7 +190,7 @@ static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type)
/* TODO: remove this global var.. it's so ugly */
nr_cpus = 1;
- return uffd_test_ctx_init(test->uffd_feature_required);
+ return uffd_test_ctx_init(test->uffd_feature_required, errmsg);
}
static bool uffd_feature_supported(uffd_test_case_t *test)
@@ -838,6 +839,7 @@ int main(int argc, char *argv[])
uffd_test_case_t *test;
mem_type_t *mem_type;
char test_name[128];
+ const char *errmsg;
int has_uffd;
int i, j;
@@ -863,8 +865,8 @@ int main(int argc, char *argv[])
uffd_test_skip("feature missing");
continue;
}
- if (uffd_setup_environment(test, mem_type)) {
- uffd_test_skip("memory allocation failed");
+ if (uffd_setup_environment(test, mem_type, &errmsg)) {
+ uffd_test_skip(errmsg);
continue;
}
test->uffd_fn();
--
2.39.1
Move it over and make it split into two tests, one for pagemap and one for
the new WP_UNPOPULATED (to be a separate one).
The thp pagemap test wasn't really working (with MADV_HUGEPAGE). Let's
just drop it (since it never really worked anyway..) and leave that for
later.
Signed-off-by: Peter Xu <[email protected]>
---
tools/testing/selftests/mm/uffd-stress.c | 166 -------------------
tools/testing/selftests/mm/uffd-unit-tests.c | 145 ++++++++++++++++
2 files changed, 145 insertions(+), 166 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 70cb0619354e..50738a993afc 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -671,157 +671,6 @@ static int userfaultfd_minor_test(void)
return stats.missing_faults != 0 || stats.minor_faults != nr_pages;
}
-static int pagemap_open(void)
-{
- int fd = open("/proc/self/pagemap", O_RDONLY);
-
- if (fd < 0)
- err("open pagemap");
-
- return fd;
-}
-
-/* This macro let __LINE__ works in err() */
-#define pagemap_check_wp(value, wp) do { \
- if (!!(value & PM_UFFD_WP) != wp) \
- err("pagemap uffd-wp bit error: 0x%"PRIx64, value); \
- } while (0)
-
-static int pagemap_test_fork(bool present)
-{
- pid_t child = fork();
- uint64_t value;
- int fd, result;
-
- if (!child) {
- /* Open the pagemap fd of the child itself */
- fd = pagemap_open();
- value = pagemap_get_entry(fd, area_dst);
- /*
- * After fork() uffd-wp bit should be gone as long as we're
- * without UFFD_FEATURE_EVENT_FORK
- */
- pagemap_check_wp(value, false);
- /* Succeed */
- exit(0);
- }
- waitpid(child, &result, 0);
- return result;
-}
-
-static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
-{
- uint64_t value;
-
- /* Test applying pte marker to anon unpopulated */
- wp_range(uffd, (uint64_t)area_dst, page_size, true);
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, true);
-
- /* Test unprotect on anon pte marker */
- wp_range(uffd, (uint64_t)area_dst, page_size, false);
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
-
- /* Test zap on anon marker */
- wp_range(uffd, (uint64_t)area_dst, page_size, true);
- if (madvise(area_dst, page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
-
- /* Test fault in after marker removed */
- *area_dst = 1;
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
- /* Drop it to make pte none again */
- if (madvise(area_dst, page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
-
- /* Test read-zero-page upon pte marker */
- wp_range(uffd, (uint64_t)area_dst, page_size, true);
- *(volatile char *)area_dst;
- /* Drop it to make pte none again */
- if (madvise(area_dst, page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
-}
-
-static void userfaultfd_pagemap_test(unsigned int test_pgsize)
-{
- int pagemap_fd;
- uint64_t value;
-
- /* Pagemap tests uffd-wp only */
- if (!test_uffdio_wp)
- return;
-
- /* Not enough memory to test this page size */
- if (test_pgsize > nr_pages * page_size)
- return;
-
- printf("testing uffd-wp with pagemap (pgsize=%u): ", test_pgsize);
- /* Flush so it doesn't flush twice in parent/child later */
- fflush(stdout);
-
- uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
-
- if (test_pgsize > page_size) {
- /* This is a thp test */
- if (madvise(area_dst, nr_pages * page_size, MADV_HUGEPAGE))
- err("madvise(MADV_HUGEPAGE) failed");
- } else if (test_pgsize == page_size) {
- /* This is normal page test; force no thp */
- if (madvise(area_dst, nr_pages * page_size, MADV_NOHUGEPAGE))
- err("madvise(MADV_NOHUGEPAGE) failed");
- }
-
- if (uffd_register(uffd, area_dst, nr_pages * page_size,
- false, true, false))
- err("register failed");
-
- pagemap_fd = pagemap_open();
-
- /* Smoke test WP_UNPOPULATED first when it's still empty */
- if (test_pgsize == page_size)
- userfaultfd_wp_unpopulated_test(pagemap_fd);
-
- /* Touch the page */
- *area_dst = 1;
- wp_range(uffd, (uint64_t)area_dst, test_pgsize, true);
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, true);
- /* Make sure uffd-wp bit dropped when fork */
- if (pagemap_test_fork(true))
- err("Detected stall uffd-wp bit in child");
-
- /* Exclusive required or PAGEOUT won't work */
- if (!(value & PM_MMAP_EXCLUSIVE))
- err("multiple mapping detected: 0x%"PRIx64, value);
-
- if (madvise(area_dst, test_pgsize, MADV_PAGEOUT))
- err("madvise(MADV_PAGEOUT) failed");
-
- /* Uffd-wp should persist even swapped out */
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, true);
- /* Make sure uffd-wp bit dropped when fork */
- if (pagemap_test_fork(false))
- err("Detected stall uffd-wp bit in child");
-
- /* Unprotect; this tests swap pte modifications */
- wp_range(uffd, (uint64_t)area_dst, page_size, false);
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
-
- /* Fault in the page from disk */
- *area_dst = 2;
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
-
- close(pagemap_fd);
- printf("done\n");
-}
-
static int userfaultfd_stress(void)
{
void *area;
@@ -933,21 +782,6 @@ static int userfaultfd_stress(void)
uffd_stats_report(uffd_stats, nr_cpus);
}
- if (test_type == TEST_ANON) {
- /*
- * shmem/hugetlb won't be able to run since they have different
- * behavior on fork() (file-backed memory normally drops ptes
- * directly when fork), meanwhile the pagemap test will verify
- * pgtable entry of fork()ed child.
- */
- userfaultfd_pagemap_test(page_size);
- /*
- * Hard-code for x86_64 for now for 2M THP, as x86_64 is
- * currently the only one that supports uffd-wp
- */
- userfaultfd_pagemap_test(page_size * 512);
- }
-
return userfaultfd_zeropage_test() || userfaultfd_sig_test()
|| userfaultfd_events_test() || userfaultfd_minor_test();
}
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 007145063363..bcd67bd4ec90 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -200,7 +200,152 @@ static bool uffd_feature_supported(uffd_test_case_t *test)
test->uffd_feature_required;
}
+static int pagemap_open(void)
+{
+ int fd = open("/proc/self/pagemap", O_RDONLY);
+
+ if (fd < 0)
+ err("open pagemap");
+
+ return fd;
+}
+
+/* This macro let __LINE__ works in err() */
+#define pagemap_check_wp(value, wp) do { \
+ if (!!(value & PM_UFFD_WP) != wp) \
+ err("pagemap uffd-wp bit error: 0x%"PRIx64, value); \
+ } while (0)
+
+static int pagemap_test_fork(bool present)
+{
+ pid_t child = fork();
+ uint64_t value;
+ int fd, result;
+
+ if (!child) {
+ /* Open the pagemap fd of the child itself */
+ fd = pagemap_open();
+ value = pagemap_get_entry(fd, area_dst);
+ /*
+ * After fork() uffd-wp bit should be gone as long as we're
+ * without UFFD_FEATURE_EVENT_FORK
+ */
+ pagemap_check_wp(value, false);
+ /* Succeed */
+ exit(0);
+ }
+ waitpid(child, &result, 0);
+ return result;
+}
+
+static void uffd_wp_unpopulated_test(void)
+{
+ uint64_t value;
+ int pagemap_fd;
+
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ false, true, false))
+ err("register failed");
+
+ pagemap_fd = pagemap_open();
+
+ /* Test applying pte marker to anon unpopulated */
+ wp_range(uffd, (uint64_t)area_dst, page_size, true);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, true);
+
+ /* Test unprotect on anon pte marker */
+ wp_range(uffd, (uint64_t)area_dst, page_size, false);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+
+ /* Test zap on anon marker */
+ wp_range(uffd, (uint64_t)area_dst, page_size, true);
+ if (madvise(area_dst, page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+
+ /* Test fault in after marker removed */
+ *area_dst = 1;
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+ /* Drop it to make pte none again */
+ if (madvise(area_dst, page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+
+ /* Test read-zero-page upon pte marker */
+ wp_range(uffd, (uint64_t)area_dst, page_size, true);
+ *(volatile char *)area_dst;
+ /* Drop it to make pte none again */
+ if (madvise(area_dst, page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+
+ uffd_test_pass();
+}
+
+static void uffd_pagemap_test(void)
+{
+ int pagemap_fd;
+ uint64_t value;
+
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ false, true, false))
+ err("register failed");
+
+ pagemap_fd = pagemap_open();
+
+ /* Touch the page */
+ *area_dst = 1;
+ wp_range(uffd, (uint64_t)area_dst, page_size, true);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, true);
+ /* Make sure uffd-wp bit dropped when fork */
+ if (pagemap_test_fork(true))
+ err("Detected stall uffd-wp bit in child");
+
+ /* Exclusive required or PAGEOUT won't work */
+ if (!(value & PM_MMAP_EXCLUSIVE))
+ err("multiple mapping detected: 0x%"PRIx64, value);
+
+ if (madvise(area_dst, page_size, MADV_PAGEOUT))
+ err("madvise(MADV_PAGEOUT) failed");
+
+ /* Uffd-wp should persist even swapped out */
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, true);
+ /* Make sure uffd-wp bit dropped when fork */
+ if (pagemap_test_fork(false))
+ err("Detected stall uffd-wp bit in child");
+
+ /* Unprotect; this tests swap pte modifications */
+ wp_range(uffd, (uint64_t)area_dst, page_size, false);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+
+ /* Fault in the page from disk */
+ *area_dst = 2;
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+
+ close(pagemap_fd);
+ uffd_test_pass();
+}
+
uffd_test_case_t uffd_tests[] = {
+ {
+ .name = "pagemap",
+ .uffd_fn = uffd_pagemap_test,
+ .mem_targets = MEM_ANON,
+ .uffd_feature_required = UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ },
+ {
+ .name = "wp-unpopulated",
+ .uffd_fn = uffd_wp_unpopulated_test,
+ .mem_targets = MEM_ANON,
+ .uffd_feature_required =
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP | UFFD_FEATURE_WP_UNPOPULATED,
+ },
};
int main(int argc, char *argv[])
--
2.39.1
On 30.03.23 17:56, Peter Xu wrote:
> This is a proposal to revert commit 914eedcb9ba0ff53c33808.
>
> I found this when writting a simple UFFDIO_API test to be the first unit
> test in this set. Two things breaks with the commit:
>
> - UFFDIO_API check was lost and missing. According to man page, the
> kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
> check is needed if the api version will be extended in the future, or
> user app won't be able to identify which is a new kernel.
Agreed.
>
> - Feature flags checks were removed, which means UFFDIO_API with a
> feature that does not exist will also succeed. According to the man
> page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
> unknown features passed in.
>
Agreed.
I understand the motivation of the original commit, but it should not
have changed existing checks/functionality. Introducing a different way
to enable such functionality on explicit request would be better. But
maybe simple feature probing (is X support? is Y supported? is Z
supported) might be easier without requiring ABI changes.
I assume we better add
Fixes: 914eedcb9ba0 ("userfaultfd: don't fail on unrecognized features")
Acked-by: David Hildenbrand <[email protected]>
> Link: https://lore.kernel.org/r/[email protected]
> Cc: Axel Rasmussen <[email protected]>
> Cc: linux-stable <[email protected]>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> fs/userfaultfd.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
> index 8395605790f6..3b2a41c330e6 100644
> --- a/fs/userfaultfd.c
> +++ b/fs/userfaultfd.c
> @@ -1977,8 +1977,10 @@ static int userfaultfd_api(struct userfaultfd_ctx *ctx,
> ret = -EFAULT;
> if (copy_from_user(&uffdio_api, buf, sizeof(uffdio_api)))
> goto out;
> - /* Ignore unsupported features (userspace built against newer kernel) */
> - features = uffdio_api.features & UFFD_API_FEATURES;
> + features = uffdio_api.features;
> + ret = -EINVAL;
> + if (uffdio_api.api != UFFD_API || (features & ~UFFD_API_FEATURES))
> + goto err_out;
> ret = -EPERM;
> if ((features & UFFD_FEATURE_EVENT_FORK) && !capable(CAP_SYS_PTRACE))
> goto err_out;
--
Thanks,
David / dhildenb
On Thu, Mar 30, 2023 at 8:57 AM Peter Xu <[email protected]> wrote:
>
> This is a proposal to revert commit 914eedcb9ba0ff53c33808.
>
> I found this when writting a simple UFFDIO_API test to be the first unit
> test in this set. Two things breaks with the commit:
>
> - UFFDIO_API check was lost and missing. According to man page, the
> kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
> check is needed if the api version will be extended in the future, or
> user app won't be able to identify which is a new kernel.
100% agreed, this was a mistake.
>
> - Feature flags checks were removed, which means UFFDIO_API with a
> feature that does not exist will also succeed. According to the man
> page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
> unknown features passed in.
I still strongly disagree with reverting this part, my feeling is
still that doing so makes things more complicated for no reason.
Re: David's point, it's clearly wrong to change semantics so a thing
that used to work now fails. But this instead makes it more permissive
- existing userspace programs continue to work as-is, but *also* one
can achieve the same thing more simply (combine probing +
configuration into one step). I don't see any problem with that,
generally.
But, if David and others don't find my argument convincing, it isn't
the end of the world. It just means I have to go update my userspace
code to be a bit more complicated. :)
I also still think the man page is incorrect or at least incomplete no
matter what we do here; we should be sure to update it as a follow-up.
>
> Link: https://lore.kernel.org/r/[email protected]
> Cc: Axel Rasmussen <[email protected]>
> Cc: linux-stable <[email protected]>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> fs/userfaultfd.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
> index 8395605790f6..3b2a41c330e6 100644
> --- a/fs/userfaultfd.c
> +++ b/fs/userfaultfd.c
> @@ -1977,8 +1977,10 @@ static int userfaultfd_api(struct userfaultfd_ctx *ctx,
> ret = -EFAULT;
> if (copy_from_user(&uffdio_api, buf, sizeof(uffdio_api)))
> goto out;
> - /* Ignore unsupported features (userspace built against newer kernel) */
> - features = uffdio_api.features & UFFD_API_FEATURES;
> + features = uffdio_api.features;
> + ret = -EINVAL;
> + if (uffdio_api.api != UFFD_API || (features & ~UFFD_API_FEATURES))
> + goto err_out;
> ret = -EPERM;
> if ((features & UFFD_FEATURE_EVENT_FORK) && !capable(CAP_SYS_PTRACE))
> goto err_out;
> --
> 2.39.1
>
On Thu, Mar 30, 2023 at 9:06 AM Peter Xu <[email protected]> wrote:
>
> Dump a summary after running whatever test specified. Useful for human
> runners to identify any kind of failures (besides exit code).
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index c0f93b668c0c..9cc33984aa9f 100644
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -5,6 +5,9 @@
> # Kselftest framework requirement - SKIP code is 4.
> ksft_skip=4
>
> +count_pass=0
> +count_fail=0
> +count_skip=0
> exitcode=0
>
> usage() {
> @@ -149,11 +152,14 @@ run_test() {
> "$@"
> local ret=$?
> if [ $ret -eq 0 ]; then
> + count_pass=$(( $count_pass + 1 ))
Actually, inside $(( )) there's no need to prefix variable names with
$ too. Running "shellcheck" over the script written this way will
generate a warning.
Same applies below.
> echo "[PASS]"
> elif [ $ret -eq $ksft_skip ]; then
> + count_skip=$(( $count_skip + 1 ))
> echo "[SKIP]"
> exitcode=$ksft_skip
> else
> + count_fail=$(( $count_fail + 1 ))
> echo "[FAIL]"
> exitcode=1
> fi
> @@ -279,4 +285,6 @@ CATEGORY="soft_dirty" run_test ./soft-dirty
> # COW tests
> CATEGORY="cow" run_test ./cow
>
> +echo "SUMMARY: PASS=${count_pass} SKIP=${count_skip} FAIL=${count_fail}"
> +
> exit $exitcode
> --
> 2.39.1
>
Besides the nitpick:
Reviewed-by: Axel Rasmussen <[email protected]>
On Thu, Mar 30, 2023 at 9:07 AM Peter Xu <[email protected]> wrote:
>
> There're two util headers under mm/ kselftest. Merge one with another. It
> turns out util.h is the easy one to move.
>
> When merging, drop PAGE_SIZE / PAGE_SHIFT because they're unnecessary
> wrappers to page_size() / page_shift(), meanwhile rename them to psize()
> and pshift() so as to not conflict with some existing definitions in some
> test files that includes vm_util.h.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Axel Rasmussen <[email protected]>
> ---
> tools/testing/selftests/mm/Makefile | 4 ++
> tools/testing/selftests/mm/gup_test.c | 5 +-
> tools/testing/selftests/mm/ksm_tests.c | 2 +-
> tools/testing/selftests/mm/mrelease_test.c | 11 ++-
> tools/testing/selftests/mm/transhuge-stress.c | 12 ++--
> tools/testing/selftests/mm/util.h | 69 -------------------
> tools/testing/selftests/mm/vm_util.c | 31 +++++++++
> tools/testing/selftests/mm/vm_util.h | 31 +++++++++
> 8 files changed, 80 insertions(+), 85 deletions(-)
> delete mode 100644 tools/testing/selftests/mm/util.h
>
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index fbf5646b1072..4188435967ed 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -111,6 +111,10 @@ $(OUTPUT)/madv_populate: vm_util.c
> $(OUTPUT)/soft-dirty: vm_util.c
> $(OUTPUT)/split_huge_page_test: vm_util.c
> $(OUTPUT)/userfaultfd: vm_util.c
> +$(OUTPUT)/gup_test: vm_util.c
> +$(OUTPUT)/mrelease_test: vm_util.c
> +$(OUTPUT)/transhuge-stress: vm_util.c
> +$(OUTPUT)/ksm_tests: vm_util.c
>
> ifeq ($(MACHINE),x86_64)
> BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
> diff --git a/tools/testing/selftests/mm/gup_test.c b/tools/testing/selftests/mm/gup_test.c
> index e43879291dac..ec2229136384 100644
> --- a/tools/testing/selftests/mm/gup_test.c
> +++ b/tools/testing/selftests/mm/gup_test.c
> @@ -12,8 +12,7 @@
> #include <assert.h>
> #include <mm/gup_test.h>
> #include "../kselftest.h"
> -
> -#include "util.h"
> +#include "vm_util.h"
>
> #define MB (1UL << 20)
>
> @@ -251,7 +250,7 @@ int main(int argc, char **argv)
> if (touch) {
> gup.gup_flags |= FOLL_TOUCH;
> } else {
> - for (; (unsigned long)p < gup.addr + size; p += PAGE_SIZE)
> + for (; (unsigned long)p < gup.addr + size; p += psize())
> p[0] = 0;
> }
>
> diff --git a/tools/testing/selftests/mm/ksm_tests.c b/tools/testing/selftests/mm/ksm_tests.c
> index 9fb21b982dc9..85a49aea3ab8 100644
> --- a/tools/testing/selftests/mm/ksm_tests.c
> +++ b/tools/testing/selftests/mm/ksm_tests.c
> @@ -14,7 +14,7 @@
>
> #include "../kselftest.h"
> #include <include/vdso/time64.h>
> -#include "util.h"
> +#include "vm_util.h"
>
> #define KSM_SYSFS_PATH "/sys/kernel/mm/ksm/"
> #define KSM_FP(s) (KSM_SYSFS_PATH s)
> diff --git a/tools/testing/selftests/mm/mrelease_test.c b/tools/testing/selftests/mm/mrelease_test.c
> index 6c62966ab5db..37b6d33b9e84 100644
> --- a/tools/testing/selftests/mm/mrelease_test.c
> +++ b/tools/testing/selftests/mm/mrelease_test.c
> @@ -9,8 +9,7 @@
> #include <stdlib.h>
> #include <sys/wait.h>
> #include <unistd.h>
> -
> -#include "util.h"
> +#include "vm_util.h"
>
> #include "../kselftest.h"
>
> @@ -32,7 +31,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
> unsigned long i;
> char *buf;
>
> - buf = (char *)mmap(NULL, nr_pages * PAGE_SIZE, PROT_READ | PROT_WRITE,
> + buf = (char *)mmap(NULL, nr_pages * psize(), PROT_READ | PROT_WRITE,
> MAP_PRIVATE | MAP_ANON, 0, 0);
> if (buf == MAP_FAILED) {
> perror("mmap failed, halting the test");
> @@ -40,7 +39,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
> }
>
> for (i = 0; i < nr_pages; i++)
> - *((unsigned long *)(buf + (i * PAGE_SIZE))) = i;
> + *((unsigned long *)(buf + (i * psize()))) = i;
>
> /* Signal the parent that the child is ready */
> if (write(pipefd, "", 1) < 0) {
> @@ -54,7 +53,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
> timeout--;
> }
>
> - munmap(buf, nr_pages * PAGE_SIZE);
> + munmap(buf, nr_pages * psize());
>
> return (timeout > 0) ? KSFT_PASS : KSFT_FAIL;
> }
> @@ -87,7 +86,7 @@ static int child_main(int pipefd[], size_t size)
>
> /* Allocate and fault-in memory and wait to be killed */
> close(pipefd[0]);
> - res = alloc_noexit(MB(size) / PAGE_SIZE, pipefd[1]);
> + res = alloc_noexit(MB(size) / psize(), pipefd[1]);
> close(pipefd[1]);
> return res;
> }
> diff --git a/tools/testing/selftests/mm/transhuge-stress.c b/tools/testing/selftests/mm/transhuge-stress.c
> index e3f00adb1b82..ba9d37ad3a89 100644
> --- a/tools/testing/selftests/mm/transhuge-stress.c
> +++ b/tools/testing/selftests/mm/transhuge-stress.c
> @@ -15,7 +15,7 @@
> #include <fcntl.h>
> #include <string.h>
> #include <sys/mman.h>
> -#include "util.h"
> +#include "vm_util.h"
>
> int backing_fd = -1;
> int mmap_flags = MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE;
> @@ -34,10 +34,10 @@ int main(int argc, char **argv)
> int pagemap_fd;
>
> ram = sysconf(_SC_PHYS_PAGES);
> - if (ram > SIZE_MAX / sysconf(_SC_PAGESIZE) / 4)
> + if (ram > SIZE_MAX / psize() / 4)
> ram = SIZE_MAX / 4;
> else
> - ram *= sysconf(_SC_PAGESIZE);
> + ram *= psize();
> len = ram;
>
> while (++i < argc) {
> @@ -58,7 +58,7 @@ int main(int argc, char **argv)
>
> warnx("allocate %zd transhuge pages, using %zd MiB virtual memory"
> " and %zd MiB of ram", len >> HPAGE_SHIFT, len >> 20,
> - ram >> (20 + HPAGE_SHIFT - PAGE_SHIFT - 1));
> + ram >> (20 + HPAGE_SHIFT - pshift() - 1));
>
> pagemap_fd = open("/proc/self/pagemap", O_RDONLY);
> if (pagemap_fd < 0)
> @@ -92,7 +92,7 @@ int main(int argc, char **argv)
> if (pfn < 0) {
> nr_failed++;
> } else {
> - size_t idx = pfn >> (HPAGE_SHIFT - PAGE_SHIFT);
> + size_t idx = pfn >> (HPAGE_SHIFT - pshift());
>
> nr_succeed++;
> if (idx >= map_len) {
> @@ -108,7 +108,7 @@ int main(int argc, char **argv)
> }
>
> /* split transhuge page, keep last page */
> - if (madvise(p, HPAGE_SIZE - PAGE_SIZE, MADV_DONTNEED))
> + if (madvise(p, HPAGE_SIZE - psize(), MADV_DONTNEED))
> err(2, "MADV_DONTNEED");
> }
> clock_gettime(CLOCK_MONOTONIC, &b);
> diff --git a/tools/testing/selftests/mm/util.h b/tools/testing/selftests/mm/util.h
> deleted file mode 100644
> index b27d26199334..000000000000
> --- a/tools/testing/selftests/mm/util.h
> +++ /dev/null
> @@ -1,69 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -
> -#ifndef __KSELFTEST_VM_UTIL_H
> -#define __KSELFTEST_VM_UTIL_H
> -
> -#include <stdint.h>
> -#include <sys/mman.h>
> -#include <err.h>
> -#include <string.h> /* ffsl() */
> -#include <unistd.h> /* _SC_PAGESIZE */
> -
> -static unsigned int __page_size;
> -static unsigned int __page_shift;
> -
> -static inline unsigned int page_size(void)
> -{
> - if (!__page_size)
> - __page_size = sysconf(_SC_PAGESIZE);
> - return __page_size;
> -}
> -
> -static inline unsigned int page_shift(void)
> -{
> - if (!__page_shift)
> - __page_shift = (ffsl(page_size()) - 1);
> - return __page_shift;
> -}
> -
> -#define PAGE_SHIFT (page_shift())
> -#define PAGE_SIZE (page_size())
> -/*
> - * On ppc64 this will only work with radix 2M hugepage size
> - */
> -#define HPAGE_SHIFT 21
> -#define HPAGE_SIZE (1 << HPAGE_SHIFT)
> -
> -#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
> -#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
> -
> -
> -static inline int64_t allocate_transhuge(void *ptr, int pagemap_fd)
> -{
> - uint64_t ent[2];
> -
> - /* drop pmd */
> - if (mmap(ptr, HPAGE_SIZE, PROT_READ | PROT_WRITE,
> - MAP_FIXED | MAP_ANONYMOUS |
> - MAP_NORESERVE | MAP_PRIVATE, -1, 0) != ptr)
> - errx(2, "mmap transhuge");
> -
> - if (madvise(ptr, HPAGE_SIZE, MADV_HUGEPAGE))
> - err(2, "MADV_HUGEPAGE");
> -
> - /* allocate transparent huge page */
> - *(volatile void **)ptr = ptr;
> -
> - if (pread(pagemap_fd, ent, sizeof(ent),
> - (uintptr_t)ptr >> (PAGE_SHIFT - 3)) != sizeof(ent))
> - err(2, "read pagemap");
> -
> - if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
> - PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
> - !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - PAGE_SHIFT)) - 1)))
> - return PAGEMAP_PFN(ent[0]);
> -
> - return -1;
> -}
> -
> -#endif
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 40e795624ff3..0204c469be43 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -8,6 +8,9 @@
> #define SMAP_FILE_PATH "/proc/self/smaps"
> #define MAX_LINE_LENGTH 500
>
> +unsigned int __page_size;
> +unsigned int __page_shift;
> +
> uint64_t pagemap_get_entry(int fd, char *start)
> {
> const unsigned long pfn = (unsigned long)start / getpagesize();
> @@ -149,3 +152,31 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size)
> {
> return __check_huge(addr, "ShmemPmdMapped:", nr_hpages, hpage_size);
> }
> +
> +int64_t allocate_transhuge(void *ptr, int pagemap_fd)
> +{
> + uint64_t ent[2];
> +
> + /* drop pmd */
> + if (mmap(ptr, HPAGE_SIZE, PROT_READ | PROT_WRITE,
> + MAP_FIXED | MAP_ANONYMOUS |
> + MAP_NORESERVE | MAP_PRIVATE, -1, 0) != ptr)
> + errx(2, "mmap transhuge");
> +
> + if (madvise(ptr, HPAGE_SIZE, MADV_HUGEPAGE))
> + err(2, "MADV_HUGEPAGE");
> +
> + /* allocate transparent huge page */
> + *(volatile void **)ptr = ptr;
> +
> + if (pread(pagemap_fd, ent, sizeof(ent),
> + (uintptr_t)ptr >> (pshift() - 3)) != sizeof(ent))
> + err(2, "read pagemap");
> +
> + if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
> + PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
> + !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - pshift())) - 1)))
> + return PAGEMAP_PFN(ent[0]);
> +
> + return -1;
> +}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index 1995ee911ef2..6edeb531afc6 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -1,6 +1,27 @@
> /* SPDX-License-Identifier: GPL-2.0 */
> #include <stdint.h>
> #include <stdbool.h>
> +#include <sys/mman.h>
> +#include <err.h>
> +#include <string.h> /* ffsl() */
> +#include <unistd.h> /* _SC_PAGESIZE */
> +
> +extern unsigned int __page_size;
> +extern unsigned int __page_shift;
> +
> +static inline unsigned int psize(void)
> +{
> + if (!__page_size)
> + __page_size = sysconf(_SC_PAGESIZE);
> + return __page_size;
> +}
> +
> +static inline unsigned int pshift(void)
> +{
> + if (!__page_shift)
> + __page_shift = (ffsl(psize()) - 1);
> + return __page_shift;
> +}
>
> uint64_t pagemap_get_entry(int fd, char *start);
> bool pagemap_is_softdirty(int fd, char *start);
> @@ -13,3 +34,13 @@ uint64_t read_pmd_pagesize(void);
> bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);
> bool check_huge_file(void *addr, int nr_hpages, uint64_t hpage_size);
> bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
> +int64_t allocate_transhuge(void *ptr, int pagemap_fd);
> +
> +/*
> + * On ppc64 this will only work with radix 2M hugepage size
> + */
> +#define HPAGE_SHIFT 21
> +#define HPAGE_SIZE (1 << HPAGE_SHIFT)
> +
> +#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
> +#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
> --
> 2.39.1
>
On Thu, Mar 30, 2023 at 9:07 AM Peter Xu <[email protected]> wrote:
>
> There're already 3 same definitions of the three functions. Move it into
> vm_util.[ch].
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Axel Rasmussen <[email protected]>
> ---
> tools/testing/selftests/mm/hugetlb-madvise.c | 25 +-------------------
> tools/testing/selftests/mm/thuge-gen.c | 19 +--------------
> tools/testing/selftests/mm/userfaultfd.c | 24 -------------------
> tools/testing/selftests/mm/vm_util.c | 21 ++++++++++++++++
> tools/testing/selftests/mm/vm_util.h | 1 +
> 5 files changed, 24 insertions(+), 66 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/hugetlb-madvise.c b/tools/testing/selftests/mm/hugetlb-madvise.c
> index 9a127a8fe176..28426e30d9bc 100644
> --- a/tools/testing/selftests/mm/hugetlb-madvise.c
> +++ b/tools/testing/selftests/mm/hugetlb-madvise.c
> @@ -18,6 +18,7 @@
> #include <unistd.h>
> #include <sys/mman.h>
> #include <fcntl.h>
> +#include "vm_util.h"
>
> #define MIN_FREE_PAGES 20
> #define NR_HUGE_PAGES 10 /* common number of pages to map/allocate */
> @@ -35,30 +36,6 @@
> unsigned long huge_page_size;
> unsigned long base_page_size;
>
> -/*
> - * default_huge_page_size copied from mlock2-tests.c
> - */
> -unsigned long default_huge_page_size(void)
> -{
> - unsigned long hps = 0;
> - char *line = NULL;
> - size_t linelen = 0;
> - FILE *f = fopen("/proc/meminfo", "r");
> -
> - if (!f)
> - return 0;
> - while (getline(&line, &linelen, f) > 0) {
> - if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
> - hps <<= 10;
> - break;
> - }
> - }
> -
> - free(line);
> - fclose(f);
> - return hps;
> -}
> -
> unsigned long get_free_hugepages(void)
> {
> unsigned long fhp = 0;
> diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
> index 361ef7192cc6..380ab5f0a534 100644
> --- a/tools/testing/selftests/mm/thuge-gen.c
> +++ b/tools/testing/selftests/mm/thuge-gen.c
> @@ -24,6 +24,7 @@
> #include <unistd.h>
> #include <stdarg.h>
> #include <string.h>
> +#include "vm_util.h"
>
> #define err(x) perror(x), exit(1)
>
> @@ -74,24 +75,6 @@ void find_pagesizes(void)
> globfree(&g);
> }
>
> -unsigned long default_huge_page_size(void)
> -{
> - unsigned long hps = 0;
> - char *line = NULL;
> - size_t linelen = 0;
> - FILE *f = fopen("/proc/meminfo", "r");
> - if (!f)
> - return 0;
> - while (getline(&line, &linelen, f) > 0) {
> - if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
> - hps <<= 10;
> - break;
> - }
> - }
> - free(line);
> - return hps;
> -}
> -
> void show(unsigned long ps)
> {
> char buf[100];
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index a96d126cb40e..4cc80a0e8955 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -1703,30 +1703,6 @@ static int userfaultfd_stress(void)
> || userfaultfd_events_test() || userfaultfd_minor_test();
> }
>
> -/*
> - * Copied from mlock2-tests.c
> - */
> -unsigned long default_huge_page_size(void)
> -{
> - unsigned long hps = 0;
> - char *line = NULL;
> - size_t linelen = 0;
> - FILE *f = fopen("/proc/meminfo", "r");
> -
> - if (!f)
> - return 0;
> - while (getline(&line, &linelen, f) > 0) {
> - if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
> - hps <<= 10;
> - break;
> - }
> - }
> -
> - free(line);
> - fclose(f);
> - return hps;
> -}
> -
> static void set_test_type(const char *type)
> {
> if (!strcmp(type, "anon")) {
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 0204c469be43..12dc654b5be3 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -180,3 +180,24 @@ int64_t allocate_transhuge(void *ptr, int pagemap_fd)
>
> return -1;
> }
> +
> +unsigned long default_huge_page_size(void)
> +{
> + unsigned long hps = 0;
> + char *line = NULL;
> + size_t linelen = 0;
> + FILE *f = fopen("/proc/meminfo", "r");
> +
> + if (!f)
> + return 0;
> + while (getline(&line, &linelen, f) > 0) {
> + if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
> + hps <<= 10;
> + break;
> + }
> + }
> +
> + free(line);
> + fclose(f);
> + return hps;
> +}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index 6edeb531afc6..d7163fff8fb7 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -35,6 +35,7 @@ bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);
> bool check_huge_file(void *addr, int nr_hpages, uint64_t hpage_size);
> bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
> int64_t allocate_transhuge(void *ptr, int pagemap_fd);
> +unsigned long default_huge_page_size(void);
>
> /*
> * On ppc64 this will only work with radix 2M hugepage size
> --
> 2.39.1
>
On Thu, Mar 30, 2023 at 9:07 AM Peter Xu <[email protected]> wrote:
>
> Meanwhile drop pagemap_read_vaddr().
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Axel Rasmussen <[email protected]>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 31 +++++++-----------------
> 1 file changed, 9 insertions(+), 22 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index 7e841f7e2884..795fbc4d84f8 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -1399,19 +1399,6 @@ static int pagemap_open(void)
> return fd;
> }
>
> -static uint64_t pagemap_read_vaddr(int fd, void *vaddr)
> -{
> - uint64_t value;
> - int ret;
> -
> - ret = pread(fd, &value, sizeof(uint64_t),
> - ((uint64_t)vaddr >> 12) * sizeof(uint64_t));
> - if (ret != sizeof(uint64_t))
> - err("pread() on pagemap failed");
> -
> - return value;
> -}
> -
> /* This macro let __LINE__ works in err() */
> #define pagemap_check_wp(value, wp) do { \
> if (!!(value & PM_UFFD_WP) != wp) \
> @@ -1427,7 +1414,7 @@ static int pagemap_test_fork(bool present)
> if (!child) {
> /* Open the pagemap fd of the child itself */
> fd = pagemap_open();
> - value = pagemap_read_vaddr(fd, area_dst);
> + value = pagemap_get_entry(fd, area_dst);
> /*
> * After fork() uffd-wp bit should be gone as long as we're
> * without UFFD_FEATURE_EVENT_FORK
> @@ -1446,24 +1433,24 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
>
> /* Test applying pte marker to anon unpopulated */
> wp_range(uffd, (uint64_t)area_dst, page_size, true);
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, true);
>
> /* Test unprotect on anon pte marker */
> wp_range(uffd, (uint64_t)area_dst, page_size, false);
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
>
> /* Test zap on anon marker */
> wp_range(uffd, (uint64_t)area_dst, page_size, true);
> if (madvise(area_dst, page_size, MADV_DONTNEED))
> err("madvise(MADV_DONTNEED) failed");
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
>
> /* Test fault in after marker removed */
> *area_dst = 1;
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
> /* Drop it to make pte none again */
> if (madvise(area_dst, page_size, MADV_DONTNEED))
> @@ -1522,7 +1509,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
> /* Touch the page */
> *area_dst = 1;
> wp_range(uffd, (uint64_t)area_dst, test_pgsize, true);
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, true);
> /* Make sure uffd-wp bit dropped when fork */
> if (pagemap_test_fork(true))
> @@ -1536,7 +1523,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
> err("madvise(MADV_PAGEOUT) failed");
>
> /* Uffd-wp should persist even swapped out */
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, true);
> /* Make sure uffd-wp bit dropped when fork */
> if (pagemap_test_fork(false))
> @@ -1544,12 +1531,12 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
>
> /* Unprotect; this tests swap pte modifications */
> wp_range(uffd, (uint64_t)area_dst, page_size, false);
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
>
> /* Fault in the page from disk */
> *area_dst = 2;
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
>
> close(pagemap_fd);
> --
> 2.39.1
>
On Thu, Mar 30, 2023 at 08:31:30PM +0200, David Hildenbrand wrote:
> On 30.03.23 17:56, Peter Xu wrote:
> > This is a proposal to revert commit 914eedcb9ba0ff53c33808.
> >
> > I found this when writting a simple UFFDIO_API test to be the first unit
> > test in this set. Two things breaks with the commit:
> >
> > - UFFDIO_API check was lost and missing. According to man page, the
> > kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
> > check is needed if the api version will be extended in the future, or
> > user app won't be able to identify which is a new kernel.
>
> Agreed.
>
> >
> > - Feature flags checks were removed, which means UFFDIO_API with a
> > feature that does not exist will also succeed. According to the man
> > page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
> > unknown features passed in.
> >
>
> Agreed.
>
> I understand the motivation of the original commit, but it should not have
> changed existing checks/functionality. Introducing a different way to enable
> such functionality on explicit request would be better. But maybe simple
> feature probing (is X support? is Y supported? is Z supported) might be
> easier without requiring ABI changes.
Yes, I mentioned a similar "proposal" of UFFDIO_FEATURES here too, simply
returning the feature bitmask before UFFDIO_API:
https://lore.kernel.org/all/ZCSUTSbAcwBINiNk@x1n/
But I think current way is still fine; so maybe we'd just not bother.
>
> I assume we better add
>
> Fixes: 914eedcb9ba0 ("userfaultfd: don't fail on unrecognized features")
Yes I'll add it.
> Acked-by: David Hildenbrand <[email protected]>
Thanks,
--
Peter Xu
On Thu, Mar 30, 2023 at 12:04:09PM -0700, Axel Rasmussen wrote:
> On Thu, Mar 30, 2023 at 8:57 AM Peter Xu <[email protected]> wrote:
> >
> > This is a proposal to revert commit 914eedcb9ba0ff53c33808.
> >
> > I found this when writting a simple UFFDIO_API test to be the first unit
> > test in this set. Two things breaks with the commit:
> >
> > - UFFDIO_API check was lost and missing. According to man page, the
> > kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
> > check is needed if the api version will be extended in the future, or
> > user app won't be able to identify which is a new kernel.
>
> 100% agreed, this was a mistake.
>
> >
> > - Feature flags checks were removed, which means UFFDIO_API with a
> > feature that does not exist will also succeed. According to the man
> > page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
> > unknown features passed in.
>
> I still strongly disagree with reverting this part, my feeling is
> still that doing so makes things more complicated for no reason.
>
> Re: David's point, it's clearly wrong to change semantics so a thing
> that used to work now fails. But this instead makes it more permissive
> - existing userspace programs continue to work as-is, but *also* one
> can achieve the same thing more simply (combine probing +
> configuration into one step). I don't see any problem with that,
> generally.
>
> But, if David and others don't find my argument convincing, it isn't
> the end of the world. It just means I have to go update my userspace
> code to be a bit more complicated. :)
I'd say it's fine if it's your own program that you can in full control
easily. :) Sorry again for not noticing that earlier.
There's one reason that we may consider keeping the behavior. IMHO it is
when there're major softwares that uses the "wrong" ABI (let's say so;
because it's not following the man pages). If you're aware any such major
softwares (especially open sourced) will break due to this revert patch,
please shoot.
>
> I also still think the man page is incorrect or at least incomplete no
> matter what we do here; we should be sure to update it as a follow-up.
So far it looks still fine if with this revert. Maybe I overlooked
somewhere?
I'll add this into my todo, but with low priority. If you have suggestion
already on how to improve the man page please do so before me!
--
Peter Xu
On Thu, Mar 30, 2023 at 12:07:24PM -0700, Axel Rasmussen wrote:
> On Thu, Mar 30, 2023 at 9:06 AM Peter Xu <[email protected]> wrote:
> >
> > Dump a summary after running whatever test specified. Useful for human
> > runners to identify any kind of failures (besides exit code).
> >
> > Signed-off-by: Peter Xu <[email protected]>
> > ---
> > tools/testing/selftests/mm/run_vmtests.sh | 8 ++++++++
> > 1 file changed, 8 insertions(+)
> >
> > diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> > index c0f93b668c0c..9cc33984aa9f 100644
> > --- a/tools/testing/selftests/mm/run_vmtests.sh
> > +++ b/tools/testing/selftests/mm/run_vmtests.sh
> > @@ -5,6 +5,9 @@
> > # Kselftest framework requirement - SKIP code is 4.
> > ksft_skip=4
> >
> > +count_pass=0
> > +count_fail=0
> > +count_skip=0
> > exitcode=0
> >
> > usage() {
> > @@ -149,11 +152,14 @@ run_test() {
> > "$@"
> > local ret=$?
> > if [ $ret -eq 0 ]; then
> > + count_pass=$(( $count_pass + 1 ))
>
> Actually, inside $(( )) there's no need to prefix variable names with
> $ too. Running "shellcheck" over the script written this way will
> generate a warning.
>
> Same applies below.
Sure, I'll fix.
>
> > echo "[PASS]"
> > elif [ $ret -eq $ksft_skip ]; then
> > + count_skip=$(( $count_skip + 1 ))
> > echo "[SKIP]"
> > exitcode=$ksft_skip
> > else
> > + count_fail=$(( $count_fail + 1 ))
> > echo "[FAIL]"
> > exitcode=1
> > fi
> > @@ -279,4 +285,6 @@ CATEGORY="soft_dirty" run_test ./soft-dirty
> > # COW tests
> > CATEGORY="cow" run_test ./cow
> >
> > +echo "SUMMARY: PASS=${count_pass} SKIP=${count_skip} FAIL=${count_fail}"
> > +
> > exit $exitcode
> > --
> > 2.39.1
> >
>
> Besides the nitpick:
>
> Reviewed-by: Axel Rasmussen <[email protected]>
Thanks!
--
Peter Xu
On Thu, Mar 30, 2023 at 11:56:38AM -0400, Peter Xu wrote:
> UFFDIO_API (with syscall)... done
> UFFDIO_API (with /dev/userfaultfd)... done
> zeropage on anon... done
> zeropage on shmem... done
> zeropage on shmem-private... done
> zeropage-hugetlb on hugetlb... done
> zeropage-hugetlb on hugetlb-private... done
> pagemap on anon... pagemap on anon... pagemap on anon... done
> wp-unpopulated on anon... done
> minor on shmem... done
> minor on hugetlb... done
> minor-wp on shmem... done
> minor-wp on hugetlb... done
> minor-collapse on shmem... done
> sigbus on anon... sigbus on anon... done
> sigbus on shmem... sigbus on shmem... done
> sigbus on shmem-private... sigbus on shmem-private... done
> sigbus on hugetlb... sigbus on hugetlb... done
> sigbus on hugetlb-private... sigbus on hugetlb-private... done
> sigbus-wp on anon... sigbus-wp on anon... done
> sigbus-wp on shmem... sigbus-wp on shmem... done
> sigbus-wp on shmem-private... sigbus-wp on shmem-private... done
> sigbus-wp on hugetlb... sigbus-wp on hugetlb... done
> sigbus-wp on hugetlb-private... sigbus-wp on hugetlb-private... done
> events on anon... events on anon... done
> events on shmem... events on shmem... done
> events on shmem-private... events on shmem-private... done
> events on hugetlb... events on hugetlb... done
> events on hugetlb-private... events on hugetlb-private... done
> events-wp on anon... events-wp on anon... done
> events-wp on shmem... events-wp on shmem... done
> events-wp on shmem-private... events-wp on shmem-private... done
> events-wp on hugetlb... events-wp on hugetlb... done
> events-wp on hugetlb-private... events-wp on hugetlb-private... done
Oops, the sigbus/events test do not look right here.. I think I slightly
messed it up right before I post this set.
I'll squash below change into "selftests/mm: UFFDIO_API test" in the next
post to fix this up:
===8<===
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 9e7f7c7f2982..793931da5056 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -92,8 +92,10 @@ static void uffd_test_pass(void)
}
#define uffd_test_start(...) do { \
+ printf("Testing "); \
printf(__VA_ARGS__); \
printf("... "); \
+ fflush(stdout); \
uffd_test_acct.total++; \
} while (0)
===8<===
So it should look like:
===8<===
Testing UFFDIO_API (with syscall)... done
Testing UFFDIO_API (with /dev/userfaultfd)... done
Testing zeropage on anon... done
Testing zeropage on shmem... done
Testing zeropage on shmem-private... done
Testing zeropage-hugetlb on hugetlb... done
Testing zeropage-hugetlb on hugetlb-private... done
Testing pagemap on anon... done
Testing wp-unpopulated on anon... done
Testing minor on shmem... done
Testing minor on hugetlb... done
Testing minor-wp on shmem... done
Testing minor-wp on hugetlb... done
Testing minor-collapse on shmem... done
Testing sigbus on anon... done
Testing sigbus on shmem... done
Testing sigbus on shmem-private... done
Testing sigbus on hugetlb... done
Testing sigbus on hugetlb-private... done
Testing sigbus-wp on anon... done
Testing sigbus-wp on shmem... done
Testing sigbus-wp on shmem-private... done
Testing sigbus-wp on hugetlb... done
Testing sigbus-wp on hugetlb-private... done
Testing events on anon... done
Testing events on shmem... done
Testing events on shmem-private... done
Testing events on hugetlb... done
Testing events on hugetlb-private... done
Testing events-wp on anon... done
Testing events-wp on shmem... done
Testing events-wp on shmem-private... done
Testing events-wp on hugetlb... done
Testing events-wp on hugetlb-private... done
Userfaults unit tests: pass=34, skip=0, fail=0 (total=34)
===8<===
--
Peter Xu
On Thu, Mar 30, 2023 at 3:27 PM Peter Xu <[email protected]> wrote:
>
> On Thu, Mar 30, 2023 at 12:04:09PM -0700, Axel Rasmussen wrote:
> > On Thu, Mar 30, 2023 at 8:57 AM Peter Xu <[email protected]> wrote:
> > >
> > > This is a proposal to revert commit 914eedcb9ba0ff53c33808.
> > >
> > > I found this when writting a simple UFFDIO_API test to be the first unit
> > > test in this set. Two things breaks with the commit:
> > >
> > > - UFFDIO_API check was lost and missing. According to man page, the
> > > kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
> > > check is needed if the api version will be extended in the future, or
> > > user app won't be able to identify which is a new kernel.
> >
> > 100% agreed, this was a mistake.
> >
> > >
> > > - Feature flags checks were removed, which means UFFDIO_API with a
> > > feature that does not exist will also succeed. According to the man
> > > page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
> > > unknown features passed in.
> >
> > I still strongly disagree with reverting this part, my feeling is
> > still that doing so makes things more complicated for no reason.
> >
> > Re: David's point, it's clearly wrong to change semantics so a thing
> > that used to work now fails. But this instead makes it more permissive
> > - existing userspace programs continue to work as-is, but *also* one
> > can achieve the same thing more simply (combine probing +
> > configuration into one step). I don't see any problem with that,
> > generally.
> >
> > But, if David and others don't find my argument convincing, it isn't
> > the end of the world. It just means I have to go update my userspace
> > code to be a bit more complicated. :)
>
> I'd say it's fine if it's your own program that you can in full control
> easily. :) Sorry again for not noticing that earlier.
>
> There's one reason that we may consider keeping the behavior. IMHO it is
> when there're major softwares that uses the "wrong" ABI (let's say so;
> because it's not following the man pages). If you're aware any such major
> softwares (especially open sourced) will break due to this revert patch,
> please shoot.
Well, I did find one example, criu:
https://github.com/checkpoint-restore/criu/blob/criu-dev/criu/uffd.c#L266
It doesn't do the two-step probing process, it looks to me like it
does what my patch was intending users to do:
It just asks for the requested features up-front, without any probing.
And then it returns the subset of features it *actually* got,
ostensibly so the caller can compare that vs. what it requested.
Then again, it looks like the caller doesn't *actually* compare the
features it got vs. what it asked for. I don't know enough about criu
to know if this is a bug, or if they actually just don't care.
https://github.com/checkpoint-restore/criu/blob/criu-dev/criu/uffd.c#L312
>
> >
> > I also still think the man page is incorrect or at least incomplete no
> > matter what we do here; we should be sure to update it as a follow-up.
>
> So far it looks still fine if with this revert. Maybe I overlooked
> somewhere?
>
> I'll add this into my todo, but with low priority. If you have suggestion
> already on how to improve the man page please do so before me!
My thinking is that it describes the bits and pieces, but doesn't
explicitly describe end-to-end how to configure a userfaultfd using
the two-step probing approach. (Or state that this is actually
*required*, unless you only want to set features=0 in any case.)
Maybe it will be easiest if I just send a patch myself with what I'm
thinking, and we can see what folks think. Always easier to just look
at a patch vs. talking about it in the abstract. :)
>
> --
> Peter Xu
>
On Fri, 31 Mar 2023 at 17:52, Axel Rasmussen <[email protected]> wrote:
>
> On Thu, Mar 30, 2023 at 3:27 PM Peter Xu <[email protected]> wrote:
> >
> > On Thu, Mar 30, 2023 at 12:04:09PM -0700, Axel Rasmussen wrote:
> > > On Thu, Mar 30, 2023 at 8:57 AM Peter Xu <[email protected]> wrote:
> > > >
> > > > This is a proposal to revert commit 914eedcb9ba0ff53c33808.
> > > >
> > > > I found this when writting a simple UFFDIO_API test to be the first unit
> > > > test in this set. Two things breaks with the commit:
> > > >
> > > > - UFFDIO_API check was lost and missing. According to man page, the
> > > > kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
> > > > check is needed if the api version will be extended in the future, or
> > > > user app won't be able to identify which is a new kernel.
> > > >
> > > > - Feature flags checks were removed, which means UFFDIO_API with a
> > > > feature that does not exist will also succeed. According to the man
> > > > page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
> > > > unknown features passed in.
If features/flags are not checked in kernel, and the kernel doesn't
return an error on
an unknown flag/error, that makes the syscall non-extendable, meaning
that adding
any new feature may break existing software, which doesn't sanitize
them properly.
https://lwn.net/Articles/588444/
See a bunch of painful exercises from syscalls with numbers in the end:
https://lwn.net/Articles/792628/
To adding an additional setsockopt() because an old one didn't have
sanity checks for flags:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8917a777be3b
(not the best example, as the new setsockopt() didn't check flags for
sanity as well (sic!),
but that's near the code I work on now)
This is even documented nowadays:
https://www.kernel.org/doc/html/latest/process/adding-syscalls.html#designing-the-api-planning-for-extension
...and everyone knows what happens when you blame userspace for breaking by
not doing what you would have expected it to do:
https://lkml.org/lkml/2012/12/23/75
[..]
> > There's one reason that we may consider keeping the behavior. IMHO it is
> > when there're major softwares that uses the "wrong" ABI (let's say so;
> > because it's not following the man pages). If you're aware any such major
> > softwares (especially open sourced) will break due to this revert patch,
> > please shoot.
>
> Well, I did find one example, criu:
> https://github.com/checkpoint-restore/criu/blob/criu-dev/criu/uffd.c#L266
Mike can speak better than me about uffd, but AFAICS, CRIU correctly detects
features with kerneldat/kdat:
https://github.com/checkpoint-restore/criu/blob/criu-dev/criu/kerndat.c#L1235
So, doing a sane thing in kernel shouldn't break CRIU (at least here).
Thanks,
Dmitry
On 03/30/23 12:07, Peter Xu wrote:
> There're already 3 same definitions of the three functions. Move it into
> vm_util.[ch].
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/hugetlb-madvise.c | 25 +-------------------
> tools/testing/selftests/mm/thuge-gen.c | 19 +--------------
> tools/testing/selftests/mm/userfaultfd.c | 24 -------------------
> tools/testing/selftests/mm/vm_util.c | 21 ++++++++++++++++
> tools/testing/selftests/mm/vm_util.h | 1 +
> 5 files changed, 24 insertions(+), 66 deletions(-)
Thanks!
Some of those copies were make by me. Sorry.
Reviewed-by: Mike Kravetz <[email protected]>
BTW - The same code (even with '... copied from mlock2-tests.c) resides
in the selftests/memfd directory. I did that as well. :( Suspect it is OK
to leave the copy there. But, it does make me wonder why memfd is not in
the mm directory?
--
Mike Kravetz
On 03/30/23 12:07, Peter Xu wrote:
> We've got the macros in uffd-stress.c, move it over and use it in
> vm_util.h.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 8 --------
> tools/testing/selftests/mm/vm_util.c | 16 ++++------------
> tools/testing/selftests/mm/vm_util.h | 8 ++++++++
> 3 files changed, 12 insertions(+), 20 deletions(-)
Nice, thanks!
Reviewed-by: Mike Kravetz <[email protected]>
--
Mike Kravetz
On 03/30/23 12:07, Peter Xu wrote:
> Meanwhile drop pagemap_read_vaddr().
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 31 +++++++-----------------
> 1 file changed, 9 insertions(+), 22 deletions(-)
Thanks,
Reviewed-by: Mike Kravetz <[email protected]>
--
Mike Kravetz
On 03/30/23 12:07, Peter Xu wrote:
> Make the check as simple as "test_type == TEST_HUGETLB" because that's the
> only mem that doesn't support ZEROPAGE.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index 795fbc4d84f8..d724f1c78847 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
> {
> struct uffdio_zeropage uffdio_zeropage;
> int ret;
> - bool has_zeropage = get_expected_ioctls(0) & (1 << _UFFDIO_ZEROPAGE);
> + bool has_zeropage = !(test_type == TEST_HUGETLB);
It is true that hugetlb is the only mem type that does not support zeropage.
So, the change is correct.
However, I actually prefer the explicit check that is there today. It seems
more like a test of the API. And, is more future proof is code changes.
Just my opinion/thoughts, not a strong objection.
--
Mike Kravetz
On Fri, Mar 31, 2023 at 11:08 AM Dmitry Safonov <[email protected]> wrote:
>
> On Fri, 31 Mar 2023 at 17:52, Axel Rasmussen <[email protected]> wrote:
> >
> > On Thu, Mar 30, 2023 at 3:27 PM Peter Xu <[email protected]> wrote:
> > >
> > > On Thu, Mar 30, 2023 at 12:04:09PM -0700, Axel Rasmussen wrote:
> > > > On Thu, Mar 30, 2023 at 8:57 AM Peter Xu <[email protected]> wrote:
> > > > >
> > > > > This is a proposal to revert commit 914eedcb9ba0ff53c33808.
> > > > >
> > > > > I found this when writting a simple UFFDIO_API test to be the first unit
> > > > > test in this set. Two things breaks with the commit:
> > > > >
> > > > > - UFFDIO_API check was lost and missing. According to man page, the
> > > > > kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
> > > > > check is needed if the api version will be extended in the future, or
> > > > > user app won't be able to identify which is a new kernel.
> > > > >
> > > > > - Feature flags checks were removed, which means UFFDIO_API with a
> > > > > feature that does not exist will also succeed. According to the man
> > > > > page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
> > > > > unknown features passed in.
>
> If features/flags are not checked in kernel, and the kernel doesn't
> return an error on
> an unknown flag/error, that makes the syscall non-extendable, meaning
> that adding
> any new feature may break existing software, which doesn't sanitize
> them properly.
> https://lwn.net/Articles/588444/
I don't think the same problem applies here. In the case of syscalls,
the problem is the only way the kernel can communicate is by the
EINVAL return value. Without the check, if a call succeeds the caller
can't tell: was the flag supported + applied, or unrecognized +
ignored?
With UFFDIO_API (we aren't talking about userfaultfd(2) itself), when
you pass in a set of flags, we return the subset of flags which were
enabled, in addition to the return code. So via that mechanism, one is
"able to check whether it is running on a kernel where [userfaultfd]
supports [the feature]" as the article describes - the only difference
is, the caller must check the returned set of features, instead of
checking for an error code. I don't think it's exactly *how* userspace
can check that's important, but rather *that* it can check.
Another important difference: I have a hard time imagining a case
where adding a new feature could break userspace, even with my
approach, but let's say for the sake of argument one arises in the
future. Unlike normal syscalls, we have the UFFD_API version check, so
we have the option of incrementing that to separate users relying on
the old behavior, from users willing to deal with the new behavior.
(Syscalls can kind of replicate this by adding a new syscall, like
clone() vs clone2(), but I think that's messier than the API version
check being built-in to the API.)
>
> See a bunch of painful exercises from syscalls with numbers in the end:
> https://lwn.net/Articles/792628/
> To adding an additional setsockopt() because an old one didn't have
> sanity checks for flags:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8917a777be3b
> (not the best example, as the new setsockopt() didn't check flags for
> sanity as well (sic!),
> but that's near the code I work on now)
>
> This is even documented nowadays:
> https://www.kernel.org/doc/html/latest/process/adding-syscalls.html#designing-the-api-planning-for-extension
>
> ...and everyone knows what happens when you blame userspace for breaking by
> not doing what you would have expected it to do:
> https://lkml.org/lkml/2012/12/23/75
100% agreed. :)
>
> [..]
> > > There's one reason that we may consider keeping the behavior. IMHO it is
> > > when there're major softwares that uses the "wrong" ABI (let's say so;
> > > because it's not following the man pages). If you're aware any such major
> > > softwares (especially open sourced) will break due to this revert patch,
> > > please shoot.
> >
> > Well, I did find one example, criu:
> > https://github.com/checkpoint-restore/criu/blob/criu-dev/criu/uffd.c#L266
>
> Mike can speak better than me about uffd, but AFAICS, CRIU correctly detects
> features with kerneldat/kdat:
> https://github.com/checkpoint-restore/criu/blob/criu-dev/criu/kerndat.c#L1235
Ah, right, this is the simplest case where no optional features are
asked for. So, it's not a great example; this particular case would
look the same regardless of what the kernel does.
>
> So, doing a sane thing in kernel shouldn't break CRIU (at least here).
>
> Thanks,
> Dmitry
On 03/30/23 12:07, Peter Xu wrote:
> The idea was trying to flip this var in the alarm handler from time to time
> to test -EEXIST of UFFDIO_ZEROPAGE, but firstly it's only used in the
> zeropage test so probably only used once, meanwhile we passed
> "retry==false" so it'll never got tested anyway.
>
> Drop both sides so we always test UFFDIO_ZEROPAGE retries if has_zeropage
> is set (!hugetlb).
>
> One more thing to do is doing UFFDIO_REGISTER for the alias buffer too,
> because otherwise the test won't even pass! We were just lucky that this
> test never really got ran at all.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 20 +++++++++++---------
> 1 file changed, 11 insertions(+), 9 deletions(-)
Thanks! With only passing "retry==false", that code is indeed dead. I had
to read the code again to understand area_dst_alias. Thanks for taking this
on as the code is difficult to understand.
Reviewed-by: Mike Kravetz <[email protected]>
--
Mike Kravetz
On Fri, Mar 31, 2023 at 11:37 AM Mike Kravetz <[email protected]> wrote:
>
> On 03/30/23 12:07, Peter Xu wrote:
> > Make the check as simple as "test_type == TEST_HUGETLB" because that's the
> > only mem that doesn't support ZEROPAGE.
> >
> > Signed-off-by: Peter Xu <[email protected]>
> > ---
> > tools/testing/selftests/mm/userfaultfd.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> > index 795fbc4d84f8..d724f1c78847 100644
> > --- a/tools/testing/selftests/mm/userfaultfd.c
> > +++ b/tools/testing/selftests/mm/userfaultfd.c
> > @@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
> > {
> > struct uffdio_zeropage uffdio_zeropage;
> > int ret;
> > - bool has_zeropage = get_expected_ioctls(0) & (1 << _UFFDIO_ZEROPAGE);
> > + bool has_zeropage = !(test_type == TEST_HUGETLB);
>
> It is true that hugetlb is the only mem type that does not support zeropage.
> So, the change is correct.
>
> However, I actually prefer the explicit check that is there today. It seems
> more like a test of the API. And, is more future proof is code changes.
>
> Just my opinion/thoughts, not a strong objection.
I agree. The existing code is more robust to future changes where we
might support or stop supporting this ioctl in some cases. It also
proves that the ioctl works, any time the API reports that it is
supported / ought to work, independent of when the *test* thinks it
should be supported.
Then again, I think this is unlikely to change in the future, so I
also agree with Mike that it's not the biggest deal.
> --
> Mike Kravetz
On 30.03.23 18:06, Peter Xu wrote:
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/.gitignore | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
> index 1f8c36a9fa10..347277f2adc3 100644
> --- a/tools/testing/selftests/mm/.gitignore
> +++ b/tools/testing/selftests/mm/.gitignore
> @@ -36,3 +36,5 @@ split_huge_page_test
> ksm_tests
> local_config.h
> local_config.mk
> +ksm_functional_tests
> +mdwe_test
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 21:04, Axel Rasmussen wrote:
> On Thu, Mar 30, 2023 at 8:57 AM Peter Xu <[email protected]> wrote:
>>
>> This is a proposal to revert commit 914eedcb9ba0ff53c33808.
>>
>> I found this when writting a simple UFFDIO_API test to be the first unit
>> test in this set. Two things breaks with the commit:
>>
>> - UFFDIO_API check was lost and missing. According to man page, the
>> kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
>> check is needed if the api version will be extended in the future, or
>> user app won't be able to identify which is a new kernel.
>
> 100% agreed, this was a mistake.
>
>>
>> - Feature flags checks were removed, which means UFFDIO_API with a
>> feature that does not exist will also succeed. According to the man
>> page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
>> unknown features passed in.
>
> I still strongly disagree with reverting this part, my feeling is
> still that doing so makes things more complicated for no reason.
>
> Re: David's point, it's clearly wrong to change semantics so a thing
> that used to work now fails. But this instead makes it more permissive
> - existing userspace programs continue to work as-is, but *also* one
> can achieve the same thing more simply (combine probing +
> configuration into one step). I don't see any problem with that,
> generally.
>
> But, if David and others don't find my argument convincing, it isn't
> the end of the world. It just means I have to go update my userspace
> code to be a bit more complicated. :)
I'd probably find it more convincing if we'd started out with that
approach ;) . User space would have to deal with the behavior of old
kernels either way already? IOW, old kernels would reject the new flags,
new kernels would not reject them but mask them out. So changing that
behavior after the effects is somewhat suboptimal IMHO ... and rather
makes things more complicated.
--
Thanks,
David / dhildenb
On 31.03.23 00:28, Peter Xu wrote:
> On Thu, Mar 30, 2023 at 12:07:24PM -0700, Axel Rasmussen wrote:
>> On Thu, Mar 30, 2023 at 9:06 AM Peter Xu <[email protected]> wrote:
>>>
>>> Dump a summary after running whatever test specified. Useful for human
>>> runners to identify any kind of failures (besides exit code).
>>>
>>> Signed-off-by: Peter Xu <[email protected]>
>>> ---
>>> tools/testing/selftests/mm/run_vmtests.sh | 8 ++++++++
>>> 1 file changed, 8 insertions(+)
>>>
>>> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
>>> index c0f93b668c0c..9cc33984aa9f 100644
>>> --- a/tools/testing/selftests/mm/run_vmtests.sh
>>> +++ b/tools/testing/selftests/mm/run_vmtests.sh
>>> @@ -5,6 +5,9 @@
>>> # Kselftest framework requirement - SKIP code is 4.
>>> ksft_skip=4
>>>
>>> +count_pass=0
>>> +count_fail=0
>>> +count_skip=0
>>> exitcode=0
>>>
>>> usage() {
>>> @@ -149,11 +152,14 @@ run_test() {
>>> "$@"
>>> local ret=$?
>>> if [ $ret -eq 0 ]; then
>>> + count_pass=$(( $count_pass + 1 ))
>>
>> Actually, inside $(( )) there's no need to prefix variable names with
>> $ too. Running "shellcheck" over the script written this way will
>> generate a warning.
>>
>> Same applies below.
>
> Sure, I'll fix.
With that
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 18:06, Peter Xu wrote:
> There're two util headers under mm/ kselftest. Merge one with another. It
> turns out util.h is the easy one to move.
>
> When merging, drop PAGE_SIZE / PAGE_SHIFT because they're unnecessary
> wrappers to page_size() / page_shift(), meanwhile rename them to psize()
> and pshift() so as to not conflict with some existing definitions in some
> test files that includes vm_util.h.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 01.04.23 03:57, Axel Rasmussen wrote:
> On Fri, Mar 31, 2023 at 11:37 AM Mike Kravetz <[email protected]> wrote:
>>
>> On 03/30/23 12:07, Peter Xu wrote:
>>> Make the check as simple as "test_type == TEST_HUGETLB" because that's the
>>> only mem that doesn't support ZEROPAGE.
>>>
>>> Signed-off-by: Peter Xu <[email protected]>
>>> ---
>>> tools/testing/selftests/mm/userfaultfd.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
>>> index 795fbc4d84f8..d724f1c78847 100644
>>> --- a/tools/testing/selftests/mm/userfaultfd.c
>>> +++ b/tools/testing/selftests/mm/userfaultfd.c
>>> @@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
>>> {
>>> struct uffdio_zeropage uffdio_zeropage;
>>> int ret;
>>> - bool has_zeropage = get_expected_ioctls(0) & (1 << _UFFDIO_ZEROPAGE);
>>> + bool has_zeropage = !(test_type == TEST_HUGETLB);
>>
>> It is true that hugetlb is the only mem type that does not support zeropage.
>> So, the change is correct.
>>
>> However, I actually prefer the explicit check that is there today. It seems
>> more like a test of the API. And, is more future proof is code changes.
>>
>> Just my opinion/thoughts, not a strong objection.
>
> I agree. The existing code is more robust to future changes where we
> might support or stop supporting this ioctl in some cases. It also
> proves that the ioctl works, any time the API reports that it is
> supported / ought to work, independent of when the *test* thinks it
> should be supported.
>
> Then again, I think this is unlikely to change in the future, so I
> also agree with Mike that it's not the biggest deal.
As there were already discussions on eventually supporting
UFFDIO_ZEROPAGE that doesn't place the shared zeropage but ... a fresh
zeropage, it might make sense to keep it as is.
--
Thanks,
David / dhildenb
On 30.03.23 18:06, Peter Xu wrote:
> TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the mm/Makefile to
> specify programs that need to build. Logically all these binaries should
> all fall into TEST_GEN_PROGS.
>
> Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can reference
> all the tests easily later.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
Yes, makes sense
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 18:07, Peter Xu wrote:
> We've got the macros in uffd-stress.c, move it over and use it in
> vm_util.h.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 18:07, Peter Xu wrote:
> There're already 3 same definitions of the three functions. Move it into
> vm_util.[ch].
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 18:07, Peter Xu wrote:
> Meanwhile drop pagemap_read_vaddr().
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 18:07, Peter Xu wrote:
> The idea was trying to flip this var in the alarm handler from time to time
> to test -EEXIST of UFFDIO_ZEROPAGE, but firstly it's only used in the
> zeropage test so probably only used once, meanwhile we passed
> "retry==false" so it'll never got tested anyway.
>
> Drop both sides so we always test UFFDIO_ZEROPAGE retries if has_zeropage
> is set (!hugetlb).
>
> One more thing to do is doing UFFDIO_REGISTER for the alias buffer too,
> because otherwise the test won't even pass! We were just lucky that this
> test never really got ran at all.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 18:07, Peter Xu wrote:
> We do have plenty of files that want to link against vm_util.c. Just make
> it simple by linking it always.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/Makefile | 12 +-----------
> 1 file changed, 1 insertion(+), 11 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 47516a78d447..b35f3eafde3c 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -105,17 +105,7 @@ TEST_FILES += va_128TBswitch.sh
>
> include ../lib.mk
>
> -$(OUTPUT)/cow: vm_util.c
> -$(OUTPUT)/khugepaged: vm_util.c
> -$(OUTPUT)/ksm_functional_tests: vm_util.c
> -$(OUTPUT)/madv_populate: vm_util.c
> -$(OUTPUT)/soft-dirty: vm_util.c
> -$(OUTPUT)/split_huge_page_test: vm_util.c
> -$(OUTPUT)/userfaultfd: vm_util.c
> -$(OUTPUT)/gup_test: vm_util.c
> -$(OUTPUT)/mrelease_test: vm_util.c
> -$(OUTPUT)/transhuge-stress: vm_util.c
> -$(OUTPUT)/ksm_tests: vm_util.c
> +$(TEST_GEN_PROGS): vm_util.c
>
> ifeq ($(MACHINE),x86_64)
> BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 18:07, Peter Xu wrote:
> Provide two helpers to open an uffd handle. Drop the error checks around
> SKIPs because it's inside an errexit() anyway, which IMHO doesn't really
> help much if the test will not continue.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 30.03.23 18:07, Peter Xu wrote:
> Add one simple test for UFFDIO_API. With that, I also added a bunch of
> small but handy helpers along the way.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-unit-tests.c | 111 ++++++++++++++++++-
> tools/testing/selftests/mm/vm_util.c | 10 ++
> 2 files changed, 120 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index 6857388783be..dfb44ffad5f5 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -9,9 +9,118 @@
>
> #ifdef __NR_userfaultfd
>
> +struct {
> + unsigned int pass, skip, fail, total;
> +} uffd_test_acct;
> +
> +static void uffd_test_report(void)
> +{
> + printf("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n",
> + uffd_test_acct.pass,
> + uffd_test_acct.skip,
> + uffd_test_acct.fail,
> + uffd_test_acct.total);
> +}
> +
> +static void uffd_test_pass(void)
> +{
> + printf("done\n");
> + uffd_test_acct.pass++;
> +}
> +
> +#define uffd_test_start(...) do { \
> + printf(__VA_ARGS__); \
> + printf("... "); \
> + uffd_test_acct.total++; \
> + } while (0)
> +
> +#define uffd_test_fail(...) do { \
> + printf("failed [reason: "); \
> + printf(__VA_ARGS__); \
> + printf("]\n"); \
> + uffd_test_acct.fail++; \
> + } while (0)
> +
> +#define uffd_test_skip(...) do { \
> + printf("skipped [reason: "); \
> + printf(__VA_ARGS__); \
> + printf("]\n"); \
> + uffd_test_acct.skip++; \
> + } while (0)
> +
There is ksft_print_msg, ksft_test_result, ksft_test_result_fail, ... do
we maybe want to convert properly to ksft while already at it?
--
Thanks,
David / dhildenb
On Fri, Mar 31, 2023 at 11:15:35AM -0700, Mike Kravetz wrote:
> On 03/30/23 12:07, Peter Xu wrote:
> > There're already 3 same definitions of the three functions. Move it into
> > vm_util.[ch].
> >
> > Signed-off-by: Peter Xu <[email protected]>
> > ---
> > tools/testing/selftests/mm/hugetlb-madvise.c | 25 +-------------------
> > tools/testing/selftests/mm/thuge-gen.c | 19 +--------------
> > tools/testing/selftests/mm/userfaultfd.c | 24 -------------------
> > tools/testing/selftests/mm/vm_util.c | 21 ++++++++++++++++
> > tools/testing/selftests/mm/vm_util.h | 1 +
> > 5 files changed, 24 insertions(+), 66 deletions(-)
>
> Thanks!
>
> Some of those copies were make by me. Sorry.
>
> Reviewed-by: Mike Kravetz <[email protected]>
Thanks!
>
> BTW - The same code (even with '... copied from mlock2-tests.c) resides
> in the selftests/memfd directory. I did that as well. :( Suspect it is OK
> to leave the copy there. But, it does make me wonder why memfd is not in
> the mm directory?
I don't know either, I suspect it was just a personal preference when it
was firstly introduced in:
commit 4f5ce5e8d7e2da3c714df8a7fa42edb9f992fc52
Author: David Herrmann <[email protected]>
Date: Fri Aug 8 14:25:32 2014 -0700
selftests: add memfd_create() + sealing tests
So I left all those alone for now. Logically they should be merged into mm/
indeed, perhaps in some future cleanups.
--
Peter Xu
On Mon, Apr 03, 2023 at 09:55:41AM +0200, David Hildenbrand wrote:
> On 01.04.23 03:57, Axel Rasmussen wrote:
> > On Fri, Mar 31, 2023 at 11:37 AM Mike Kravetz <[email protected]> wrote:
> > >
> > > On 03/30/23 12:07, Peter Xu wrote:
> > > > Make the check as simple as "test_type == TEST_HUGETLB" because that's the
> > > > only mem that doesn't support ZEROPAGE.
> > > >
> > > > Signed-off-by: Peter Xu <[email protected]>
> > > > ---
> > > > tools/testing/selftests/mm/userfaultfd.c | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> > > > index 795fbc4d84f8..d724f1c78847 100644
> > > > --- a/tools/testing/selftests/mm/userfaultfd.c
> > > > +++ b/tools/testing/selftests/mm/userfaultfd.c
> > > > @@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
> > > > {
> > > > struct uffdio_zeropage uffdio_zeropage;
> > > > int ret;
> > > > - bool has_zeropage = get_expected_ioctls(0) & (1 << _UFFDIO_ZEROPAGE);
> > > > + bool has_zeropage = !(test_type == TEST_HUGETLB);
> > >
> > > It is true that hugetlb is the only mem type that does not support zeropage.
> > > So, the change is correct.
> > >
> > > However, I actually prefer the explicit check that is there today. It seems
> > > more like a test of the API. And, is more future proof is code changes.
> > >
> > > Just my opinion/thoughts, not a strong objection.
> >
> > I agree. The existing code is more robust to future changes where we
> > might support or stop supporting this ioctl in some cases. It also
> > proves that the ioctl works, any time the API reports that it is
> > supported / ought to work, independent of when the *test* thinks it
> > should be supported.
> >
> > Then again, I think this is unlikely to change in the future, so I
> > also agree with Mike that it's not the biggest deal.
>
> As there were already discussions on eventually supporting UFFDIO_ZEROPAGE
> that doesn't place the shared zeropage but ... a fresh zeropage, it might
> make sense to keep it as is.
Thanks everyone.
So here the major goal is to drop get_expected_ioctls(), and I think it's
really unwanted here. Besides it's a blocker for split the test in a clean
way, a major reason is get_expected_ioctls() fetches "wheter we support
zeropage for this mem" from UFFD_API_RANGE_IOCTLS, rather than from the
UFFDIO_REGISTER anyway:
uint64_t get_expected_ioctls(uint64_t mode)
{
uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
if (test_type == TEST_HUGETLB)
ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
ioctls &= ~(1 << _UFFDIO_CONTINUE);
return ioctls;
}
It means it'll succeed or fail depending on what kernel we run this test
on, and also on what headers we compile the test against.
I actually mentioned some of the reasoning in a follow up patch (sorry
maybe the split here caused some confusion):
selftests/mm: uffd_[un]register()
Add two helpers to register/unregister to an uffd. Use them to drop
duplicate codes.
This patch also drops assert_expected_ioctls_present() and
get_expected_ioctls(). Reasons:
- It'll need a lot of effort to pass test_type==HUGETLB into it from the
upper, so it's the simplest way to get rid of another global var
- The ioctls returned in UFFDIO_REGISTER is hardly useful at all, because
any app can already detect kernel support on any ioctl via its
corresponding UFFD_FEATURE_*. The check here is for sanity mostly but
it's probably destined no user app will even use it.
- It's not friendly to one future goal of uffd to run on old kernels, the
problem is get_expected_ioctls() compiles against UFFD_API_RANGE_IOCTLS,
which is a value that can change depending on where the test is compiled,
rather than reflecting what the kernel underneath has. It means it'll
report false negatives on old kernels so it's against our will.
So let's make our live easier.
But I do agree that it's helpful to keep a test against
uffdio_register.ioctls in this case against _UFFDIO_ZEROPAGE, so it can be
detected dynamically. IOW, even if we would like to avoid "test !=
HUGETLB" here, at least we should like to fix that with the UFFDIO_REGISTER
results.
Here's my offer below. :)
Could I keep this patch as-is (as part of getting rid of
get_expected_ioctls() effort; I can squash this one into "selftests/mm:
uffd_[un]register()" if any of you think proper), meanwhile I'll squash a
fixup to the "move zeropage test into uffd-unit-tests" explicitly check
uffdio_register.ioctls in the same patchset? IOW, we'll have a few test
commits missing this specific ioctl test, but then we'll have a better one
dynamically detected from the kernel.
The fixup patch attached. I think it'll automatically work when someone
would like to introduce UFFDIO_ZEROPAGE to hugetlb too, another side
benefit is I merged the zeropage test into one, which does look better too.
Thanks,
--
Peter Xu
On Fri, Mar 31, 2023 at 05:03:17PM -0700, Mike Kravetz wrote:
> On 03/30/23 12:07, Peter Xu wrote:
> > The idea was trying to flip this var in the alarm handler from time to time
> > to test -EEXIST of UFFDIO_ZEROPAGE, but firstly it's only used in the
> > zeropage test so probably only used once, meanwhile we passed
> > "retry==false" so it'll never got tested anyway.
> >
> > Drop both sides so we always test UFFDIO_ZEROPAGE retries if has_zeropage
> > is set (!hugetlb).
> >
> > One more thing to do is doing UFFDIO_REGISTER for the alias buffer too,
> > because otherwise the test won't even pass! We were just lucky that this
> > test never really got ran at all.
> >
> > Signed-off-by: Peter Xu <[email protected]>
> > ---
> > tools/testing/selftests/mm/userfaultfd.c | 20 +++++++++++---------
> > 1 file changed, 11 insertions(+), 9 deletions(-)
>
> Thanks! With only passing "retry==false", that code is indeed dead. I had
> to read the code again to understand area_dst_alias. Thanks for taking this
> on as the code is difficult to understand.
Yes it's very confusing. I plan to move on the cleanup to remove all these
global variables at least in the unit tests, but I'll first see how this
one goes.
>
> Reviewed-by: Mike Kravetz <[email protected]>
Thanks,
--
Peter Xu
On Mon, Apr 03, 2023 at 09:59:50AM +0200, David Hildenbrand wrote:
> There is ksft_print_msg, ksft_test_result, ksft_test_result_fail, ... do we
> maybe want to convert properly to ksft while already at it?
Yes, I started with trying to use that but found that there're not a lot of
things that I can leverage.
Starting with ksft_set_plan() - I think this is something we call first. I
want the current unit test to skip everything if UFFD API test failed here,
then I need to feed in a dynamic number of "plan" into ksft_set_plan().
But I never know after I ran the 1st test..
I can call ksft_set_plan() later than this, but it misses a few tests which
also looks weird.
It also seems to not really help anything at all and not obvious to use.
E.g. ksft_finished() will reference ksft_plan then it'll trigger
ksft_exit_fail() but here I want to make it SKIP if the 1st test failed
simply because the kernel probably doesn't have CONFIG_USERFAULTFD.
Another example: I never figured what does x{fail|pass|skip} meant in the
header.. e.g. ksft_inc_xfail_cnt() is used nowhere so I cannot reference
either. Then I don't know when I should increase them.
In short, to make the unit test behave as expected, I figured I'll just
write these few helpers and that's good enough for this unit test. That
takes perhaps 5 min anyway and isn't hugely bad for an unit test.
Then I keep the exit code matching kselftests (KSFT_SKIP, etc.).
What I can do here, though, is at least reuse the counters, e.g:
ksft_inc_pass_cnt() / ksft_inc_fail_cnt()
There's no ksft_inc_skip_cnt() so, maybe, I can just reuse
ksft_inc_xskip_cnt() assuming that counts "skip"s?
Let me know if you have better ideas, I'll be happy to switch in that case.
Thanks,
--
Peter Xu
On 03.04.23 18:43, Peter Xu wrote:
> On Mon, Apr 03, 2023 at 09:59:50AM +0200, David Hildenbrand wrote:
>> There is ksft_print_msg, ksft_test_result, ksft_test_result_fail, ... do we
>> maybe want to convert properly to ksft while already at it?
>
> Yes, I started with trying to use that but found that there're not a lot of
> things that I can leverage.
>
> Starting with ksft_set_plan() - I think this is something we call first. I
> want the current unit test to skip everything if UFFD API test failed here,
> then I need to feed in a dynamic number of "plan" into ksft_set_plan().
> But I never know after I ran the 1st test..
In cow.c I did that. Getting the number of tests right can be
challenging indeed.
Basic "feature availability" checks would go first (is uffd even
around?), and depending on that you can set the plan.
For everything else, you can skip instead of test, so it will still be
accounted towards the plan.
>
> I can call ksft_set_plan() later than this, but it misses a few tests which
> also looks weird.
Yeah, it would be nice to simply make ksft_set_plan() optional. For
example, make ksft_print_cnts() skip the comparison if ksft_plan == 0.
At least ksft_exit_skip() handles that already in a descend way (below).
>
> It also seems to not really help anything at all and not obvious to use.
> E.g. ksft_finished() will reference ksft_plan then it'll trigger
> ksft_exit_fail() but here I want to make it SKIP if the 1st test failed
> simply because the kernel probably doesn't have CONFIG_USERFAULTFD.
You'd simply do that availability check first and then use
ksft_exit_skip() in case not available I guess.
>
> Another example: I never figured what does x{fail|pass|skip} meant in the
> header.. e.g. ksft_inc_xfail_cnt() is used nowhere so I cannot reference
> either. Then I don't know when I should increase them.
In cow.c I have the following flow:
ksft_print_header();
ksft_set_plan();
... tests ...
err = ksft_get_fail_cnt();
if (err)
ksft_exit_fail_msg();
return ksft_exit_pass();
That gives me:
# [INFO] detected THP size: 2048 KiB
# [INFO] detected hugetlb size: 2048 KiB
# [INFO] detected hugetlb size: 1048576 KiB
# [INFO] huge zeropage is enabled
TAP version 13
1..190
...
# Totals: pass:87 fail:0 xfail:0 xpass:0 skip:103 error:0
I didn't use xfail or xpass so far, but what I understood is that these
are "expected failures" and "expected passes". fail/pass/skip are
straight forward.
ksft_test_result_fail()/ksft_test_result_pass()/ksft_test_result_skip()
are used to set them.
You'd do availability checks before ksft_set_plan() and fail with a
ksft_exit_skip() if the kernel doesn't support it. Then, you'd just use
ksft_test_result_fail()/ksft_test_result_pass()/ksft_test_result_skip().
>
> In short, to make the unit test behave as expected, I figured I'll just
> write these few helpers and that's good enough for this unit test. That
> takes perhaps 5 min anyway and isn't hugely bad for an unit test.
>
> Then I keep the exit code matching kselftests (KSFT_SKIP, etc.).
>
> What I can do here, though, is at least reuse the counters, e.g:
>
> ksft_inc_pass_cnt() / ksft_inc_fail_cnt()
>
> There's no ksft_inc_skip_cnt() so, maybe, I can just reuse
> ksft_inc_xskip_cnt() assuming that counts "skip"s?
>
> Let me know if you have better ideas, I'll be happy to switch in that case.
I guess once you start manually increasing/decreasing the cnt, you might
be abusing the ksft framework indeed and are better off handling it
differently :D
--
Thanks,
David / dhildenb
On Mon, Apr 03, 2023 at 09:06:26PM +0200, David Hildenbrand wrote:
> On 03.04.23 18:43, Peter Xu wrote:
> > On Mon, Apr 03, 2023 at 09:59:50AM +0200, David Hildenbrand wrote:
> > > There is ksft_print_msg, ksft_test_result, ksft_test_result_fail, ... do we
> > > maybe want to convert properly to ksft while already at it?
> >
> > Yes, I started with trying to use that but found that there're not a lot of
> > things that I can leverage.
> >
> > Starting with ksft_set_plan() - I think this is something we call first. I
> > want the current unit test to skip everything if UFFD API test failed here,
> > then I need to feed in a dynamic number of "plan" into ksft_set_plan().
> > But I never know after I ran the 1st test..
>
> In cow.c I did that. Getting the number of tests right can be challenging
> indeed.
IMHO the major thing is not about not easy to set, it's about there's
merely no benefit I can see of having that calculated at the start of a
test.
There's one thing it can do, that is when calling ksft_finished() it can be
used to know whether all tests are run, but sadly here we're calculating
everything just to make it match.. so it loses its last purpose.. IMHO.
>
> Basic "feature availability" checks would go first (is uffd even around?),
> and depending on that you can set the plan.
>
> For everything else, you can skip instead of test, so it will still be
> accounted towards the plan.
>
> >
> > I can call ksft_set_plan() later than this, but it misses a few tests which
> > also looks weird.
>
> Yeah, it would be nice to simply make ksft_set_plan() optional. For example,
> make ksft_print_cnts() skip the comparison if ksft_plan == 0. At least
> ksft_exit_skip() handles that already in a descend way (below).
>
> >
> > It also seems to not really help anything at all and not obvious to use.
> > E.g. ksft_finished() will reference ksft_plan then it'll trigger
> > ksft_exit_fail() but here I want to make it SKIP if the 1st test failed
> > simply because the kernel probably doesn't have CONFIG_USERFAULTFD.
>
> You'd simply do that availability check first and then use ksft_exit_skip()
> in case not available I guess.
>
> >
> > Another example: I never figured what does x{fail|pass|skip} meant in the
> > header.. e.g. ksft_inc_xfail_cnt() is used nowhere so I cannot reference
> > either. Then I don't know when I should increase them.
>
> In cow.c I have the following flow:
>
> ksft_print_header();
> ksft_set_plan();
> ... tests ...
> err = ksft_get_fail_cnt();
> if (err)
> ksft_exit_fail_msg();
> return ksft_exit_pass();
>
> That gives me:
>
> # [INFO] detected THP size: 2048 KiB
> # [INFO] detected hugetlb size: 2048 KiB
> # [INFO] detected hugetlb size: 1048576 KiB
> # [INFO] huge zeropage is enabled
> TAP version 13
> 1..190
> ...
> # Totals: pass:87 fail:0 xfail:0 xpass:0 skip:103 error:0
>
>
> I didn't use xfail or xpass so far, but what I understood is that these are
> "expected failures" and "expected passes". fail/pass/skip are straight
Yes, xfail can be expressed that way, but maybe not xpass? Otherwise it's
hard to identify what's the difference between xpass and pass, because IIUC
pass also means "expected to pass".
> forward.
> ksft_test_result_fail()/ksft_test_result_pass()/ksft_test_result_skip() are
> used to set them.
>
> You'd do availability checks before ksft_set_plan() and fail with a
> ksft_exit_skip() if the kernel doesn't support it. Then, you'd just use
> ksft_test_result_fail()/ksft_test_result_pass()/ksft_test_result_skip().
>
> >
> > In short, to make the unit test behave as expected, I figured I'll just
> > write these few helpers and that's good enough for this unit test. That
> > takes perhaps 5 min anyway and isn't hugely bad for an unit test.
> >
> > Then I keep the exit code matching kselftests (KSFT_SKIP, etc.).
> >
> > What I can do here, though, is at least reuse the counters, e.g:
> >
> > ksft_inc_pass_cnt() / ksft_inc_fail_cnt()
> >
> > There's no ksft_inc_skip_cnt() so, maybe, I can just reuse
> > ksft_inc_xskip_cnt() assuming that counts "skip"s?
> >
> > Let me know if you have better ideas, I'll be happy to switch in that case.
>
> I guess once you start manually increasing/decreasing the cnt, you might be
> abusing the ksft framework indeed and are better off handling it differently
> :D
I'm serious considering that to address your comment here, to show that I'm
trying my best to use whatever can help in this test case. :) Here reusing
it would avoid a few bytes in the bss, which is still beneficial so I can
do. At least I'm sure xskip is for skipping now, so I know what to use.
PS: one other reason I didn't use the header is also because I prefer the
current output of uffd-self-tests.c. I didn't say anything because I think
it's stingy.. So let's keep this in a trivial small PS. After all, the
print format is half of what the header provides functional-wise.
I'll see what I can come up with at last in the next version, thanks David.
--
Peter Xu
On 03.04.23 22:24, Peter Xu wrote:
> On Mon, Apr 03, 2023 at 09:06:26PM +0200, David Hildenbrand wrote:
>> On 03.04.23 18:43, Peter Xu wrote:
>>> On Mon, Apr 03, 2023 at 09:59:50AM +0200, David Hildenbrand wrote:
>>>> There is ksft_print_msg, ksft_test_result, ksft_test_result_fail, ... do we
>>>> maybe want to convert properly to ksft while already at it?
>>>
>>> Yes, I started with trying to use that but found that there're not a lot of
>>> things that I can leverage.
>>>
>>> Starting with ksft_set_plan() - I think this is something we call first. I
>>> want the current unit test to skip everything if UFFD API test failed here,
>>> then I need to feed in a dynamic number of "plan" into ksft_set_plan().
>>> But I never know after I ran the 1st test..
>>
>> In cow.c I did that. Getting the number of tests right can be challenging
>> indeed.
>
> IMHO the major thing is not about not easy to set, it's about there's
> merely no benefit I can see of having that calculated at the start of a
> test.
Thinking about it, I believe I figured out why it makes sense. By
specifying upfront how many tests you intend to run, the framework can
check if you have the right number of pass/fail/skip during that test
case execution.
This requires a different way of writing tests: each test case is
supposed to trigger the exact same number of pass/fail/skip on each
possible path.
If the numbers don't add up, it could indicate a possible bug in your
tests. For example, not triggering a fail on some exit path. While your
test execution might indicate "success", there is actually a hidden
issue in your test.
I started using the framework for all new tests, because I think it's
quite nice and at least things will be a bit consistent and possibly
tests easier to maintain.
Having that said, I can understand why one might not want to use it. And
that there are some things in there that might be improved.
>
> There's one thing it can do, that is when calling ksft_finished() it can be
> used to know whether all tests are run, but sadly here we're calculating
> everything just to make it match.. so it loses its last purpose.. IMHO.
>
>>
>> Basic "feature availability" checks would go first (is uffd even around?),
>> and depending on that you can set the plan.
>>
>> For everything else, you can skip instead of test, so it will still be
>> accounted towards the plan.
>>
>>>
>>> I can call ksft_set_plan() later than this, but it misses a few tests which
>>> also looks weird.
>>
>> Yeah, it would be nice to simply make ksft_set_plan() optional. For example,
>> make ksft_print_cnts() skip the comparison if ksft_plan == 0. At least
>> ksft_exit_skip() handles that already in a descend way (below).
>>
>>>
>>> It also seems to not really help anything at all and not obvious to use.
>>> E.g. ksft_finished() will reference ksft_plan then it'll trigger
>>> ksft_exit_fail() but here I want to make it SKIP if the 1st test failed
>>> simply because the kernel probably doesn't have CONFIG_USERFAULTFD.
>>
>> You'd simply do that availability check first and then use ksft_exit_skip()
>> in case not available I guess.
>>
>>>
>>> Another example: I never figured what does x{fail|pass|skip} meant in the
>>> header.. e.g. ksft_inc_xfail_cnt() is used nowhere so I cannot reference
>>> either. Then I don't know when I should increase them.
>>
>> In cow.c I have the following flow:
>>
>> ksft_print_header();
>> ksft_set_plan();
>> ... tests ...
>> err = ksft_get_fail_cnt();
>> if (err)
>> ksft_exit_fail_msg();
>> return ksft_exit_pass();
>>
>> That gives me:
>>
>> # [INFO] detected THP size: 2048 KiB
>> # [INFO] detected hugetlb size: 2048 KiB
>> # [INFO] detected hugetlb size: 1048576 KiB
>> # [INFO] huge zeropage is enabled
>> TAP version 13
>> 1..190
>> ...
>> # Totals: pass:87 fail:0 xfail:0 xpass:0 skip:103 error:0
>>
>>
>> I didn't use xfail or xpass so far, but what I understood is that these are
>> "expected failures" and "expected passes". fail/pass/skip are straight
>
> Yes, xfail can be expressed that way, but maybe not xpass? Otherwise it's
> hard to identify what's the difference between xpass and pass, because IIUC
> pass also means "expected to pass".
>
I'm a simple man, I use pass/fail/skip in my tests. But let's try
figuring that out; a quick internet search (no idea how trustworthy)
tells me that I was wrong about xpass:
xfail: expected failure
xpass: unexpected pass
it essentially is:
xfail -> pass
xpass -> fail
but with a slight semantic difference when thinking about a test case:
ret = mmap(0, 0 ...);
if (ret == MAP_FAILED) {
XFAIL();
} else {
XPASS();
}
vs.
ret = mmap(0, 0 ...);
if (ret == MAP_FAILED) {
PASS();
} else {
FAIL();
}
It's all inherited from other testing frameworks I think. And xpass
seems to be completely unused and xfail mostly unused.
So I wouldn't worry about that and simply use pass/fail/skip.
>> forward.
>> ksft_test_result_fail()/ksft_test_result_pass()/ksft_test_result_skip() are
>> used to set them.
>>
>> You'd do availability checks before ksft_set_plan() and fail with a
>> ksft_exit_skip() if the kernel doesn't support it. Then, you'd just use
>> ksft_test_result_fail()/ksft_test_result_pass()/ksft_test_result_skip().
>>
>>>
>>> In short, to make the unit test behave as expected, I figured I'll just
>>> write these few helpers and that's good enough for this unit test. That
>>> takes perhaps 5 min anyway and isn't hugely bad for an unit test.
>>>
>>> Then I keep the exit code matching kselftests (KSFT_SKIP, etc.).
>>>
>>> What I can do here, though, is at least reuse the counters, e.g:
>>>
>>> ksft_inc_pass_cnt() / ksft_inc_fail_cnt()
>>>
>>> There's no ksft_inc_skip_cnt() so, maybe, I can just reuse
>>> ksft_inc_xskip_cnt() assuming that counts "skip"s?
>>>
>>> Let me know if you have better ideas, I'll be happy to switch in that case.
>>
>> I guess once you start manually increasing/decreasing the cnt, you might be
>> abusing the ksft framework indeed and are better off handling it differently
>> :D
>
> I'm serious considering that to address your comment here, to show that I'm
> trying my best to use whatever can help in this test case. :) Here reusing
:) appreciated, but don't let my comments distract you. If you don't
think ksft is a good fit (or any good), then don't use it.
--
Thanks,
David / dhildenb
On Thu, Mar 30, 2023 at 12:07:52PM -0400, Peter Xu wrote:
> +int uffd_open(unsigned int flags)
> +{
> + int uffd = uffd_open_sys(flags);
> +
> + if (uffd < 0)
> + uffd = uffd_open_dev(flags);
> +
> + return uffd;
> +}
A (benign) accident when rebasing here.. I'll move this function into
"selftests/mm: Add framework for uffd-unit-test" which is its first usage.
--
Peter Xu
On Thu, Mar 30, 2023 at 12:07:47PM -0400, Peter Xu wrote:
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index c57757c2a36f..17f2bb82c3db 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -330,33 +330,6 @@ void uffd_test_ctx_init(uint64_t features)
> err("pipe");
> }
>
> -uint64_t get_expected_ioctls(uint64_t mode)
> -{
> - uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
> -
> - if (test_type == TEST_HUGETLB)
> - ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
> -
> - if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
> - ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
> -
> - if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
> - ioctls &= ~(1 << _UFFDIO_CONTINUE);
> -
> - return ioctls;
> -}
> -
> -void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
> -{
> - uint64_t expected = get_expected_ioctls(mode);
> - uint64_t actual = ioctls & expected;
> -
> - if (actual != expected) {
> - err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
> - expected, actual);
> - }
> -}
Here I dropped the other reference of get_expected_ioctls(), so I also
dropped this test which I think is kind of flawed IMHO - as I replied in
the other thread, we should probably not reference UFFD_API_RANGE_IOCTLS.
But I can feel (from the comments in the other patch that removed the other
reference of get_expected_ioctls()) that a lot of us would still care about
this test.
So I added a new patch / test on top of the series (so it'll have one more
patch in the next version at last), just to test all possible combinations
of UFFDIO_REGISTER alongside with its returned uffdio_register.ioctls.
This is IMHO better than get_expected_ioctls() because:
- It's much cleaner to have a separate test on this rather than testing
it randomly in the code with random values passed in.
- It tests all combinations. It not only includes shmem-private that this
series introduced while wasn't there before, but also all combinations
of (miss, wp, minor) tuples.
- It doesn't rely on UFFD_API_RANGE_IOCTLS anymore.
It'll be something like this:
===8<===
/*
* Test the returned uffdio_register.ioctls with different register modes.
* Note that _UFFDIO_ZEROPAGE is tested separately in the zeropage test.
*/
static void
do_register_ioctls_test(uffd_test_args_t *args, bool miss, bool wp, bool minor)
{
uint64_t ioctls = 0, expected = BIT_ULL(_UFFDIO_WAKE);
mem_type_t *mem_type = args->mem_type;
int ret;
ret = uffd_register_with_ioctls(uffd, area_dst, page_size,
miss, wp, minor, &ioctls);
/*
* Handle special cases of UFFDIO_REGISTER here where it should
* just fail with -EINVAL first..
*
* Case 1: register MINOR on anon
* Case 2: register with no mode selected
*/
if ((minor && (mem_type->mem_flag == MEM_ANON)) ||
(!miss && !wp && !minor)) {
if (ret != -EINVAL)
err("register (miss=%d, wp=%d, minor=%d) failed "
"with wrong errno=%d", miss, wp, minor, ret);
return;
}
/* UFFDIO_REGISTER should succeed, then check ioctls returned */
if (miss)
expected |= BIT_ULL(_UFFDIO_COPY);
if (wp)
expected |= BIT_ULL(_UFFDIO_WRITEPROTECT);
if (minor)
expected |= BIT_ULL(_UFFDIO_CONTINUE);
if ((ioctls & expected) != expected)
err("unexpected uffdio_register.ioctls "
"(miss=%d, wp=%d, minor=%d): expected=0x%"PRIx64", "
"returned=0x%"PRIx64, miss, wp, minor, expected, ioctls);
uffd_unregister(uffd, area_dst, page_size);
}
static void uffd_register_ioctls_test(uffd_test_args_t *args)
{
int miss, wp, minor;
for (miss = 0; miss <= 1; miss++)
for (wp = 0; wp <= 1; wp++)
for (minor = 0; minor <= 1; minor++)
do_register_ioctls_test(args, miss, wp, minor);
uffd_test_pass();
}
===8<===
Side note: the _UFFDIO_ZEROPAGE test will be left in the specific zeropage
test.
I considered moving get_expected_ioctls() rather than dropping in the same
patch, but that's just over-complicated when without the unit test
frameworks being ready. I hope this addresses the concern here, otherwise
please shoot.
I've also attached the two patches that will test uffdio_register.ioctls as
a whole, just in case helpful for discussion before I post v2.
Thanks,
--
Peter Xu
On Thu, Mar 30, 2023 at 12:06:43PM -0400, Peter Xu wrote:
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/.gitignore | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
> index 1f8c36a9fa10..347277f2adc3 100644
> --- a/tools/testing/selftests/mm/.gitignore
> +++ b/tools/testing/selftests/mm/.gitignore
> @@ -36,3 +36,5 @@ split_huge_page_test
> ksm_tests
> local_config.h
> local_config.mk
> +ksm_functional_tests
> +mdwe_test
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:06:46PM -0400, Peter Xu wrote:
> Dump a summary after running whatever test specified. Useful for human
> runners to identify any kind of failures (besides exit code).
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index c0f93b668c0c..9cc33984aa9f 100644
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -5,6 +5,9 @@
> # Kselftest framework requirement - SKIP code is 4.
> ksft_skip=4
>
> +count_pass=0
> +count_fail=0
> +count_skip=0
> exitcode=0
>
> usage() {
> @@ -149,11 +152,14 @@ run_test() {
> "$@"
> local ret=$?
> if [ $ret -eq 0 ]; then
> + count_pass=$(( $count_pass + 1 ))
> echo "[PASS]"
> elif [ $ret -eq $ksft_skip ]; then
> + count_skip=$(( $count_skip + 1 ))
> echo "[SKIP]"
> exitcode=$ksft_skip
> else
> + count_fail=$(( $count_fail + 1 ))
> echo "[FAIL]"
> exitcode=1
> fi
> @@ -279,4 +285,6 @@ CATEGORY="soft_dirty" run_test ./soft-dirty
> # COW tests
> CATEGORY="cow" run_test ./cow
>
> +echo "SUMMARY: PASS=${count_pass} SKIP=${count_skip} FAIL=${count_fail}"
> +
> exit $exitcode
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:00PM -0400, Peter Xu wrote:
> We do have plenty of files that want to link against vm_util.c. Just make
> it simple by linking it always.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/Makefile | 12 +-----------
> 1 file changed, 1 insertion(+), 11 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 47516a78d447..b35f3eafde3c 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -105,17 +105,7 @@ TEST_FILES += va_128TBswitch.sh
>
> include ../lib.mk
>
> -$(OUTPUT)/cow: vm_util.c
> -$(OUTPUT)/khugepaged: vm_util.c
> -$(OUTPUT)/ksm_functional_tests: vm_util.c
> -$(OUTPUT)/madv_populate: vm_util.c
> -$(OUTPUT)/soft-dirty: vm_util.c
> -$(OUTPUT)/split_huge_page_test: vm_util.c
> -$(OUTPUT)/userfaultfd: vm_util.c
> -$(OUTPUT)/gup_test: vm_util.c
> -$(OUTPUT)/mrelease_test: vm_util.c
> -$(OUTPUT)/transhuge-stress: vm_util.c
> -$(OUTPUT)/ksm_tests: vm_util.c
> +$(TEST_GEN_PROGS): vm_util.c
>
> ifeq ($(MACHINE),x86_64)
> BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:06:54PM -0400, Peter Xu wrote:
> TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the mm/Makefile to
> specify programs that need to build. Logically all these binaries should
> all fall into TEST_GEN_PROGS.
>
> Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can reference
> all the tests easily later.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/Makefile | 63 +++++++++++++++--------------
> 1 file changed, 32 insertions(+), 31 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 4188435967ed..47516a78d447 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -31,34 +31,35 @@ MAKEFLAGS += --no-builtin-rules
>
> CFLAGS = -Wall -I $(top_srcdir) -I $(top_srcdir)/tools/include/uapi $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
> LDLIBS = -lrt -lpthread
> -TEST_GEN_FILES = cow
> -TEST_GEN_FILES += compaction_test
> -TEST_GEN_FILES += gup_test
> -TEST_GEN_FILES += hmm-tests
> -TEST_GEN_FILES += hugetlb-madvise
> -TEST_GEN_FILES += hugepage-mmap
> -TEST_GEN_FILES += hugepage-mremap
> -TEST_GEN_FILES += hugepage-shm
> -TEST_GEN_FILES += hugepage-vmemmap
> -TEST_GEN_FILES += khugepaged
> +
> +TEST_GEN_PROGS = cow
> +TEST_GEN_PROGS += compaction_test
> +TEST_GEN_PROGS += gup_test
> +TEST_GEN_PROGS += hmm-tests
> +TEST_GEN_PROGS += hugetlb-madvise
> +TEST_GEN_PROGS += hugepage-mmap
> +TEST_GEN_PROGS += hugepage-mremap
> +TEST_GEN_PROGS += hugepage-shm
> +TEST_GEN_PROGS += hugepage-vmemmap
> +TEST_GEN_PROGS += khugepaged
> TEST_GEN_PROGS = madv_populate
> -TEST_GEN_FILES += map_fixed_noreplace
> -TEST_GEN_FILES += map_hugetlb
> -TEST_GEN_FILES += map_populate
> -TEST_GEN_FILES += memfd_secret
> -TEST_GEN_FILES += migration
> -TEST_GEN_FILES += mlock-random-test
> -TEST_GEN_FILES += mlock2-tests
> -TEST_GEN_FILES += mrelease_test
> -TEST_GEN_FILES += mremap_dontunmap
> -TEST_GEN_FILES += mremap_test
> -TEST_GEN_FILES += on-fault-limit
> -TEST_GEN_FILES += thuge-gen
> -TEST_GEN_FILES += transhuge-stress
> -TEST_GEN_FILES += userfaultfd
> +TEST_GEN_PROGS += map_fixed_noreplace
> +TEST_GEN_PROGS += map_hugetlb
> +TEST_GEN_PROGS += map_populate
> +TEST_GEN_PROGS += memfd_secret
> +TEST_GEN_PROGS += migration
> +TEST_GEN_PROGS += mlock-random-test
> +TEST_GEN_PROGS += mlock2-tests
> +TEST_GEN_PROGS += mrelease_test
> +TEST_GEN_PROGS += mremap_dontunmap
> +TEST_GEN_PROGS += mremap_test
> +TEST_GEN_PROGS += on-fault-limit
> +TEST_GEN_PROGS += thuge-gen
> +TEST_GEN_PROGS += transhuge-stress
> +TEST_GEN_PROGS += userfaultfd
> TEST_GEN_PROGS += soft-dirty
> TEST_GEN_PROGS += split_huge_page_test
> -TEST_GEN_FILES += ksm_tests
> +TEST_GEN_PROGS += ksm_tests
> TEST_GEN_PROGS += ksm_functional_tests
> TEST_GEN_PROGS += mdwe_test
>
> @@ -76,24 +77,24 @@ CFLAGS += -no-pie
> endif
>
> ifeq ($(CAN_BUILD_I386),1)
> -TEST_GEN_FILES += $(BINARIES_32)
> +TEST_GEN_PROGS += $(BINARIES_32)
> endif
>
> ifeq ($(CAN_BUILD_X86_64),1)
> -TEST_GEN_FILES += $(BINARIES_64)
> +TEST_GEN_PROGS += $(BINARIES_64)
> endif
> else
>
> ifneq (,$(findstring $(MACHINE),ppc64))
> -TEST_GEN_FILES += protection_keys
> +TEST_GEN_PROGS += protection_keys
> endif
>
> endif
>
> ifneq (,$(filter $(MACHINE),arm64 ia64 mips64 parisc64 ppc64 riscv64 s390x sh64 sparc64 x86_64))
> -TEST_GEN_FILES += va_128TBswitch
> -TEST_GEN_FILES += virtual_address_range
> -TEST_GEN_FILES += write_to_hugetlbfs
> +TEST_GEN_PROGS += va_128TBswitch
> +TEST_GEN_PROGS += virtual_address_range
> +TEST_GEN_PROGS += write_to_hugetlbfs
> endif
>
> TEST_PROGS := run_vmtests.sh
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:06:49PM -0400, Peter Xu wrote:
> There're two util headers under mm/ kselftest. Merge one with another. It
> turns out util.h is the easy one to move.
>
> When merging, drop PAGE_SIZE / PAGE_SHIFT because they're unnecessary
> wrappers to page_size() / page_shift(), meanwhile rename them to psize()
> and pshift() so as to not conflict with some existing definitions in some
> test files that includes vm_util.h.
For me this sounds like two patches, but it's not really important with
diffstat that small.
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/Makefile | 4 ++
> tools/testing/selftests/mm/gup_test.c | 5 +-
> tools/testing/selftests/mm/ksm_tests.c | 2 +-
> tools/testing/selftests/mm/mrelease_test.c | 11 ++-
> tools/testing/selftests/mm/transhuge-stress.c | 12 ++--
> tools/testing/selftests/mm/util.h | 69 -------------------
> tools/testing/selftests/mm/vm_util.c | 31 +++++++++
> tools/testing/selftests/mm/vm_util.h | 31 +++++++++
> 8 files changed, 80 insertions(+), 85 deletions(-)
> delete mode 100644 tools/testing/selftests/mm/util.h
>
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index fbf5646b1072..4188435967ed 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -111,6 +111,10 @@ $(OUTPUT)/madv_populate: vm_util.c
> $(OUTPUT)/soft-dirty: vm_util.c
> $(OUTPUT)/split_huge_page_test: vm_util.c
> $(OUTPUT)/userfaultfd: vm_util.c
> +$(OUTPUT)/gup_test: vm_util.c
> +$(OUTPUT)/mrelease_test: vm_util.c
> +$(OUTPUT)/transhuge-stress: vm_util.c
> +$(OUTPUT)/ksm_tests: vm_util.c
>
> ifeq ($(MACHINE),x86_64)
> BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
> diff --git a/tools/testing/selftests/mm/gup_test.c b/tools/testing/selftests/mm/gup_test.c
> index e43879291dac..ec2229136384 100644
> --- a/tools/testing/selftests/mm/gup_test.c
> +++ b/tools/testing/selftests/mm/gup_test.c
> @@ -12,8 +12,7 @@
> #include <assert.h>
> #include <mm/gup_test.h>
> #include "../kselftest.h"
> -
> -#include "util.h"
> +#include "vm_util.h"
>
> #define MB (1UL << 20)
>
> @@ -251,7 +250,7 @@ int main(int argc, char **argv)
> if (touch) {
> gup.gup_flags |= FOLL_TOUCH;
> } else {
> - for (; (unsigned long)p < gup.addr + size; p += PAGE_SIZE)
> + for (; (unsigned long)p < gup.addr + size; p += psize())
> p[0] = 0;
> }
>
> diff --git a/tools/testing/selftests/mm/ksm_tests.c b/tools/testing/selftests/mm/ksm_tests.c
> index 9fb21b982dc9..85a49aea3ab8 100644
> --- a/tools/testing/selftests/mm/ksm_tests.c
> +++ b/tools/testing/selftests/mm/ksm_tests.c
> @@ -14,7 +14,7 @@
>
> #include "../kselftest.h"
> #include <include/vdso/time64.h>
> -#include "util.h"
> +#include "vm_util.h"
>
> #define KSM_SYSFS_PATH "/sys/kernel/mm/ksm/"
> #define KSM_FP(s) (KSM_SYSFS_PATH s)
> diff --git a/tools/testing/selftests/mm/mrelease_test.c b/tools/testing/selftests/mm/mrelease_test.c
> index 6c62966ab5db..37b6d33b9e84 100644
> --- a/tools/testing/selftests/mm/mrelease_test.c
> +++ b/tools/testing/selftests/mm/mrelease_test.c
> @@ -9,8 +9,7 @@
> #include <stdlib.h>
> #include <sys/wait.h>
> #include <unistd.h>
> -
> -#include "util.h"
> +#include "vm_util.h"
>
> #include "../kselftest.h"
>
> @@ -32,7 +31,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
> unsigned long i;
> char *buf;
>
> - buf = (char *)mmap(NULL, nr_pages * PAGE_SIZE, PROT_READ | PROT_WRITE,
> + buf = (char *)mmap(NULL, nr_pages * psize(), PROT_READ | PROT_WRITE,
> MAP_PRIVATE | MAP_ANON, 0, 0);
> if (buf == MAP_FAILED) {
> perror("mmap failed, halting the test");
> @@ -40,7 +39,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
> }
>
> for (i = 0; i < nr_pages; i++)
> - *((unsigned long *)(buf + (i * PAGE_SIZE))) = i;
> + *((unsigned long *)(buf + (i * psize()))) = i;
>
> /* Signal the parent that the child is ready */
> if (write(pipefd, "", 1) < 0) {
> @@ -54,7 +53,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
> timeout--;
> }
>
> - munmap(buf, nr_pages * PAGE_SIZE);
> + munmap(buf, nr_pages * psize());
>
> return (timeout > 0) ? KSFT_PASS : KSFT_FAIL;
> }
> @@ -87,7 +86,7 @@ static int child_main(int pipefd[], size_t size)
>
> /* Allocate and fault-in memory and wait to be killed */
> close(pipefd[0]);
> - res = alloc_noexit(MB(size) / PAGE_SIZE, pipefd[1]);
> + res = alloc_noexit(MB(size) / psize(), pipefd[1]);
> close(pipefd[1]);
> return res;
> }
> diff --git a/tools/testing/selftests/mm/transhuge-stress.c b/tools/testing/selftests/mm/transhuge-stress.c
> index e3f00adb1b82..ba9d37ad3a89 100644
> --- a/tools/testing/selftests/mm/transhuge-stress.c
> +++ b/tools/testing/selftests/mm/transhuge-stress.c
> @@ -15,7 +15,7 @@
> #include <fcntl.h>
> #include <string.h>
> #include <sys/mman.h>
> -#include "util.h"
> +#include "vm_util.h"
>
> int backing_fd = -1;
> int mmap_flags = MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE;
> @@ -34,10 +34,10 @@ int main(int argc, char **argv)
> int pagemap_fd;
>
> ram = sysconf(_SC_PHYS_PAGES);
> - if (ram > SIZE_MAX / sysconf(_SC_PAGESIZE) / 4)
> + if (ram > SIZE_MAX / psize() / 4)
> ram = SIZE_MAX / 4;
> else
> - ram *= sysconf(_SC_PAGESIZE);
> + ram *= psize();
> len = ram;
>
> while (++i < argc) {
> @@ -58,7 +58,7 @@ int main(int argc, char **argv)
>
> warnx("allocate %zd transhuge pages, using %zd MiB virtual memory"
> " and %zd MiB of ram", len >> HPAGE_SHIFT, len >> 20,
> - ram >> (20 + HPAGE_SHIFT - PAGE_SHIFT - 1));
> + ram >> (20 + HPAGE_SHIFT - pshift() - 1));
>
> pagemap_fd = open("/proc/self/pagemap", O_RDONLY);
> if (pagemap_fd < 0)
> @@ -92,7 +92,7 @@ int main(int argc, char **argv)
> if (pfn < 0) {
> nr_failed++;
> } else {
> - size_t idx = pfn >> (HPAGE_SHIFT - PAGE_SHIFT);
> + size_t idx = pfn >> (HPAGE_SHIFT - pshift());
>
> nr_succeed++;
> if (idx >= map_len) {
> @@ -108,7 +108,7 @@ int main(int argc, char **argv)
> }
>
> /* split transhuge page, keep last page */
> - if (madvise(p, HPAGE_SIZE - PAGE_SIZE, MADV_DONTNEED))
> + if (madvise(p, HPAGE_SIZE - psize(), MADV_DONTNEED))
> err(2, "MADV_DONTNEED");
> }
> clock_gettime(CLOCK_MONOTONIC, &b);
> diff --git a/tools/testing/selftests/mm/util.h b/tools/testing/selftests/mm/util.h
> deleted file mode 100644
> index b27d26199334..000000000000
> --- a/tools/testing/selftests/mm/util.h
> +++ /dev/null
> @@ -1,69 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -
> -#ifndef __KSELFTEST_VM_UTIL_H
> -#define __KSELFTEST_VM_UTIL_H
> -
> -#include <stdint.h>
> -#include <sys/mman.h>
> -#include <err.h>
> -#include <string.h> /* ffsl() */
> -#include <unistd.h> /* _SC_PAGESIZE */
> -
> -static unsigned int __page_size;
> -static unsigned int __page_shift;
> -
> -static inline unsigned int page_size(void)
> -{
> - if (!__page_size)
> - __page_size = sysconf(_SC_PAGESIZE);
> - return __page_size;
> -}
> -
> -static inline unsigned int page_shift(void)
> -{
> - if (!__page_shift)
> - __page_shift = (ffsl(page_size()) - 1);
> - return __page_shift;
> -}
> -
> -#define PAGE_SHIFT (page_shift())
> -#define PAGE_SIZE (page_size())
> -/*
> - * On ppc64 this will only work with radix 2M hugepage size
> - */
> -#define HPAGE_SHIFT 21
> -#define HPAGE_SIZE (1 << HPAGE_SHIFT)
> -
> -#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
> -#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
> -
> -
> -static inline int64_t allocate_transhuge(void *ptr, int pagemap_fd)
> -{
> - uint64_t ent[2];
> -
> - /* drop pmd */
> - if (mmap(ptr, HPAGE_SIZE, PROT_READ | PROT_WRITE,
> - MAP_FIXED | MAP_ANONYMOUS |
> - MAP_NORESERVE | MAP_PRIVATE, -1, 0) != ptr)
> - errx(2, "mmap transhuge");
> -
> - if (madvise(ptr, HPAGE_SIZE, MADV_HUGEPAGE))
> - err(2, "MADV_HUGEPAGE");
> -
> - /* allocate transparent huge page */
> - *(volatile void **)ptr = ptr;
> -
> - if (pread(pagemap_fd, ent, sizeof(ent),
> - (uintptr_t)ptr >> (PAGE_SHIFT - 3)) != sizeof(ent))
> - err(2, "read pagemap");
> -
> - if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
> - PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
> - !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - PAGE_SHIFT)) - 1)))
> - return PAGEMAP_PFN(ent[0]);
> -
> - return -1;
> -}
> -
> -#endif
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 40e795624ff3..0204c469be43 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -8,6 +8,9 @@
> #define SMAP_FILE_PATH "/proc/self/smaps"
> #define MAX_LINE_LENGTH 500
>
> +unsigned int __page_size;
> +unsigned int __page_shift;
> +
> uint64_t pagemap_get_entry(int fd, char *start)
> {
> const unsigned long pfn = (unsigned long)start / getpagesize();
> @@ -149,3 +152,31 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size)
> {
> return __check_huge(addr, "ShmemPmdMapped:", nr_hpages, hpage_size);
> }
> +
> +int64_t allocate_transhuge(void *ptr, int pagemap_fd)
> +{
> + uint64_t ent[2];
> +
> + /* drop pmd */
> + if (mmap(ptr, HPAGE_SIZE, PROT_READ | PROT_WRITE,
> + MAP_FIXED | MAP_ANONYMOUS |
> + MAP_NORESERVE | MAP_PRIVATE, -1, 0) != ptr)
> + errx(2, "mmap transhuge");
> +
> + if (madvise(ptr, HPAGE_SIZE, MADV_HUGEPAGE))
> + err(2, "MADV_HUGEPAGE");
> +
> + /* allocate transparent huge page */
> + *(volatile void **)ptr = ptr;
> +
> + if (pread(pagemap_fd, ent, sizeof(ent),
> + (uintptr_t)ptr >> (pshift() - 3)) != sizeof(ent))
> + err(2, "read pagemap");
> +
> + if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
> + PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
> + !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - pshift())) - 1)))
> + return PAGEMAP_PFN(ent[0]);
> +
> + return -1;
> +}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index 1995ee911ef2..6edeb531afc6 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -1,6 +1,27 @@
> /* SPDX-License-Identifier: GPL-2.0 */
> #include <stdint.h>
> #include <stdbool.h>
> +#include <sys/mman.h>
> +#include <err.h>
> +#include <string.h> /* ffsl() */
> +#include <unistd.h> /* _SC_PAGESIZE */
> +
> +extern unsigned int __page_size;
> +extern unsigned int __page_shift;
> +
> +static inline unsigned int psize(void)
> +{
> + if (!__page_size)
> + __page_size = sysconf(_SC_PAGESIZE);
> + return __page_size;
> +}
> +
> +static inline unsigned int pshift(void)
> +{
> + if (!__page_shift)
> + __page_shift = (ffsl(psize()) - 1);
> + return __page_shift;
> +}
>
> uint64_t pagemap_get_entry(int fd, char *start);
> bool pagemap_is_softdirty(int fd, char *start);
> @@ -13,3 +34,13 @@ uint64_t read_pmd_pagesize(void);
> bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);
> bool check_huge_file(void *addr, int nr_hpages, uint64_t hpage_size);
> bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
> +int64_t allocate_transhuge(void *ptr, int pagemap_fd);
> +
> +/*
> + * On ppc64 this will only work with radix 2M hugepage size
> + */
> +#define HPAGE_SHIFT 21
> +#define HPAGE_SIZE (1 << HPAGE_SHIFT)
> +
> +#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
> +#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:08PM -0400, Peter Xu wrote:
> We've got the macros in uffd-stress.c, move it over and use it in
> vm_util.h.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 8 --------
> tools/testing/selftests/mm/vm_util.c | 16 ++++------------
> tools/testing/selftests/mm/vm_util.h | 8 ++++++++
> 3 files changed, 12 insertions(+), 20 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index 4cc80a0e8955..7e841f7e2884 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -1389,14 +1389,6 @@ static int userfaultfd_minor_test(void)
> return stats.missing_faults != 0 || stats.minor_faults != nr_pages;
> }
>
> -#define BIT_ULL(nr) (1ULL << (nr))
> -#define PM_SOFT_DIRTY BIT_ULL(55)
> -#define PM_MMAP_EXCLUSIVE BIT_ULL(56)
> -#define PM_UFFD_WP BIT_ULL(57)
> -#define PM_FILE BIT_ULL(61)
> -#define PM_SWAP BIT_ULL(62)
> -#define PM_PRESENT BIT_ULL(63)
> -
> static int pagemap_open(void)
> {
> int fd = open("/proc/self/pagemap", O_RDONLY);
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 12dc654b5be3..8e9da621764a 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -25,25 +25,17 @@ uint64_t pagemap_get_entry(int fd, char *start)
>
> bool pagemap_is_softdirty(int fd, char *start)
> {
> - uint64_t entry = pagemap_get_entry(fd, start);
> -
> - // Check if dirty bit (55th bit) is set
> - return entry & 0x0080000000000000ull;
> + return pagemap_get_entry(fd, start) & PM_SOFT_DIRTY;
> }
>
> bool pagemap_is_swapped(int fd, char *start)
> {
> - uint64_t entry = pagemap_get_entry(fd, start);
> -
> - return entry & 0x4000000000000000ull;
> + return pagemap_get_entry(fd, start) & PM_SWAP;
> }
>
> bool pagemap_is_populated(int fd, char *start)
> {
> - uint64_t entry = pagemap_get_entry(fd, start);
> -
> - /* Present or swapped. */
> - return entry & 0xc000000000000000ull;
> + return pagemap_get_entry(fd, start) & (PM_PRESENT | PM_SWAP);
> }
>
> unsigned long pagemap_get_pfn(int fd, char *start)
> @@ -51,7 +43,7 @@ unsigned long pagemap_get_pfn(int fd, char *start)
> uint64_t entry = pagemap_get_entry(fd, start);
>
> /* If present (63th bit), PFN is at bit 0 -- 54. */
> - if (entry & 0x8000000000000000ull)
> + if (entry & PM_PRESENT)
> return entry & 0x007fffffffffffffull;
> return -1ul;
> }
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index d7163fff8fb7..d9fadddb5c69 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -6,6 +6,14 @@
> #include <string.h> /* ffsl() */
> #include <unistd.h> /* _SC_PAGESIZE */
>
> +#define BIT_ULL(nr) (1ULL << (nr))
> +#define PM_SOFT_DIRTY BIT_ULL(55)
> +#define PM_MMAP_EXCLUSIVE BIT_ULL(56)
> +#define PM_UFFD_WP BIT_ULL(57)
> +#define PM_FILE BIT_ULL(61)
> +#define PM_SWAP BIT_ULL(62)
> +#define PM_PRESENT BIT_ULL(63)
> +
> extern unsigned int __page_size;
> extern unsigned int __page_shift;
>
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:05PM -0400, Peter Xu wrote:
> There're already 3 same definitions of the three functions. Move it into
> vm_util.[ch].
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/hugetlb-madvise.c | 25 +-------------------
> tools/testing/selftests/mm/thuge-gen.c | 19 +--------------
> tools/testing/selftests/mm/userfaultfd.c | 24 -------------------
> tools/testing/selftests/mm/vm_util.c | 21 ++++++++++++++++
> tools/testing/selftests/mm/vm_util.h | 1 +
> 5 files changed, 24 insertions(+), 66 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/hugetlb-madvise.c b/tools/testing/selftests/mm/hugetlb-madvise.c
> index 9a127a8fe176..28426e30d9bc 100644
> --- a/tools/testing/selftests/mm/hugetlb-madvise.c
> +++ b/tools/testing/selftests/mm/hugetlb-madvise.c
> @@ -18,6 +18,7 @@
> #include <unistd.h>
> #include <sys/mman.h>
> #include <fcntl.h>
> +#include "vm_util.h"
>
> #define MIN_FREE_PAGES 20
> #define NR_HUGE_PAGES 10 /* common number of pages to map/allocate */
> @@ -35,30 +36,6 @@
> unsigned long huge_page_size;
> unsigned long base_page_size;
>
> -/*
> - * default_huge_page_size copied from mlock2-tests.c
> - */
> -unsigned long default_huge_page_size(void)
> -{
> - unsigned long hps = 0;
> - char *line = NULL;
> - size_t linelen = 0;
> - FILE *f = fopen("/proc/meminfo", "r");
> -
> - if (!f)
> - return 0;
> - while (getline(&line, &linelen, f) > 0) {
> - if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
> - hps <<= 10;
> - break;
> - }
> - }
> -
> - free(line);
> - fclose(f);
> - return hps;
> -}
> -
> unsigned long get_free_hugepages(void)
> {
> unsigned long fhp = 0;
> diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
> index 361ef7192cc6..380ab5f0a534 100644
> --- a/tools/testing/selftests/mm/thuge-gen.c
> +++ b/tools/testing/selftests/mm/thuge-gen.c
> @@ -24,6 +24,7 @@
> #include <unistd.h>
> #include <stdarg.h>
> #include <string.h>
> +#include "vm_util.h"
>
> #define err(x) perror(x), exit(1)
>
> @@ -74,24 +75,6 @@ void find_pagesizes(void)
> globfree(&g);
> }
>
> -unsigned long default_huge_page_size(void)
> -{
> - unsigned long hps = 0;
> - char *line = NULL;
> - size_t linelen = 0;
> - FILE *f = fopen("/proc/meminfo", "r");
> - if (!f)
> - return 0;
> - while (getline(&line, &linelen, f) > 0) {
> - if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
> - hps <<= 10;
> - break;
> - }
> - }
> - free(line);
> - return hps;
> -}
> -
> void show(unsigned long ps)
> {
> char buf[100];
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index a96d126cb40e..4cc80a0e8955 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -1703,30 +1703,6 @@ static int userfaultfd_stress(void)
> || userfaultfd_events_test() || userfaultfd_minor_test();
> }
>
> -/*
> - * Copied from mlock2-tests.c
> - */
> -unsigned long default_huge_page_size(void)
> -{
> - unsigned long hps = 0;
> - char *line = NULL;
> - size_t linelen = 0;
> - FILE *f = fopen("/proc/meminfo", "r");
> -
> - if (!f)
> - return 0;
> - while (getline(&line, &linelen, f) > 0) {
> - if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
> - hps <<= 10;
> - break;
> - }
> - }
> -
> - free(line);
> - fclose(f);
> - return hps;
> -}
> -
> static void set_test_type(const char *type)
> {
> if (!strcmp(type, "anon")) {
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 0204c469be43..12dc654b5be3 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -180,3 +180,24 @@ int64_t allocate_transhuge(void *ptr, int pagemap_fd)
>
> return -1;
> }
> +
> +unsigned long default_huge_page_size(void)
> +{
> + unsigned long hps = 0;
> + char *line = NULL;
> + size_t linelen = 0;
> + FILE *f = fopen("/proc/meminfo", "r");
> +
> + if (!f)
> + return 0;
> + while (getline(&line, &linelen, f) > 0) {
> + if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
> + hps <<= 10;
> + break;
> + }
> + }
> +
> + free(line);
> + fclose(f);
> + return hps;
> +}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index 6edeb531afc6..d7163fff8fb7 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -35,6 +35,7 @@ bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);
> bool check_huge_file(void *addr, int nr_hpages, uint64_t hpage_size);
> bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
> int64_t allocate_transhuge(void *ptr, int pagemap_fd);
> +unsigned long default_huge_page_size(void);
>
> /*
> * On ppc64 this will only work with radix 2M hugepage size
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:11PM -0400, Peter Xu wrote:
> Subject: selftests/mm: Reuse pagemap_get_entry() in vm_util.h
Nit: selftests/mm/uffd:
> Meanwhile drop pagemap_read_vaddr().
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 31 +++++++-----------------
> 1 file changed, 9 insertions(+), 22 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index 7e841f7e2884..795fbc4d84f8 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -1399,19 +1399,6 @@ static int pagemap_open(void)
> return fd;
> }
>
> -static uint64_t pagemap_read_vaddr(int fd, void *vaddr)
> -{
> - uint64_t value;
> - int ret;
> -
> - ret = pread(fd, &value, sizeof(uint64_t),
> - ((uint64_t)vaddr >> 12) * sizeof(uint64_t));
> - if (ret != sizeof(uint64_t))
> - err("pread() on pagemap failed");
> -
> - return value;
> -}
> -
> /* This macro let __LINE__ works in err() */
> #define pagemap_check_wp(value, wp) do { \
> if (!!(value & PM_UFFD_WP) != wp) \
> @@ -1427,7 +1414,7 @@ static int pagemap_test_fork(bool present)
> if (!child) {
> /* Open the pagemap fd of the child itself */
> fd = pagemap_open();
> - value = pagemap_read_vaddr(fd, area_dst);
> + value = pagemap_get_entry(fd, area_dst);
> /*
> * After fork() uffd-wp bit should be gone as long as we're
> * without UFFD_FEATURE_EVENT_FORK
> @@ -1446,24 +1433,24 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
>
> /* Test applying pte marker to anon unpopulated */
> wp_range(uffd, (uint64_t)area_dst, page_size, true);
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, true);
>
> /* Test unprotect on anon pte marker */
> wp_range(uffd, (uint64_t)area_dst, page_size, false);
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
>
> /* Test zap on anon marker */
> wp_range(uffd, (uint64_t)area_dst, page_size, true);
> if (madvise(area_dst, page_size, MADV_DONTNEED))
> err("madvise(MADV_DONTNEED) failed");
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
>
> /* Test fault in after marker removed */
> *area_dst = 1;
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
> /* Drop it to make pte none again */
> if (madvise(area_dst, page_size, MADV_DONTNEED))
> @@ -1522,7 +1509,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
> /* Touch the page */
> *area_dst = 1;
> wp_range(uffd, (uint64_t)area_dst, test_pgsize, true);
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, true);
> /* Make sure uffd-wp bit dropped when fork */
> if (pagemap_test_fork(true))
> @@ -1536,7 +1523,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
> err("madvise(MADV_PAGEOUT) failed");
>
> /* Uffd-wp should persist even swapped out */
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, true);
> /* Make sure uffd-wp bit dropped when fork */
> if (pagemap_test_fork(false))
> @@ -1544,12 +1531,12 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
>
> /* Unprotect; this tests swap pte modifications */
> wp_range(uffd, (uint64_t)area_dst, page_size, false);
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
>
> /* Fault in the page from disk */
> *area_dst = 2;
> - value = pagemap_read_vaddr(pagemap_fd, area_dst);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> pagemap_check_wp(value, false);
>
> close(pagemap_fd);
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Mon, Apr 03, 2023 at 12:10:43PM -0400, Peter Xu wrote:
> On Mon, Apr 03, 2023 at 09:55:41AM +0200, David Hildenbrand wrote:
> > On 01.04.23 03:57, Axel Rasmussen wrote:
> > > On Fri, Mar 31, 2023 at 11:37 AM Mike Kravetz <[email protected]> wrote:
> > > >
> > > > On 03/30/23 12:07, Peter Xu wrote:
> > > > > Make the check as simple as "test_type == TEST_HUGETLB" because that's the
> > > > > only mem that doesn't support ZEROPAGE.
> > > > >
> > > > > Signed-off-by: Peter Xu <[email protected]>
> > > > > ---
> > > > > tools/testing/selftests/mm/userfaultfd.c | 2 +-
> > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> > > > > index 795fbc4d84f8..d724f1c78847 100644
> > > > > --- a/tools/testing/selftests/mm/userfaultfd.c
> > > > > +++ b/tools/testing/selftests/mm/userfaultfd.c
> > > > > @@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
> > > > > {
> > > > > struct uffdio_zeropage uffdio_zeropage;
> > > > > int ret;
> > > > > - bool has_zeropage = get_expected_ioctls(0) & (1 << _UFFDIO_ZEROPAGE);
> > > > > + bool has_zeropage = !(test_type == TEST_HUGETLB);
> > > >
> > > > It is true that hugetlb is the only mem type that does not support zeropage.
> > > > So, the change is correct.
> > > >
> > > > However, I actually prefer the explicit check that is there today. It seems
> > > > more like a test of the API. And, is more future proof is code changes.
> > > >
> > > > Just my opinion/thoughts, not a strong objection.
> > >
> > > I agree. The existing code is more robust to future changes where we
> > > might support or stop supporting this ioctl in some cases. It also
> > > proves that the ioctl works, any time the API reports that it is
> > > supported / ought to work, independent of when the *test* thinks it
> > > should be supported.
> > >
> > > Then again, I think this is unlikely to change in the future, so I
> > > also agree with Mike that it's not the biggest deal.
> >
> > As there were already discussions on eventually supporting UFFDIO_ZEROPAGE
> > that doesn't place the shared zeropage but ... a fresh zeropage, it might
> > make sense to keep it as is.
>
> Thanks everyone.
>
> So here the major goal is to drop get_expected_ioctls(), and I think it's
> really unwanted here. Besides it's a blocker for split the test in a clean
> way, a major reason is get_expected_ioctls() fetches "wheter we support
> zeropage for this mem" from UFFD_API_RANGE_IOCTLS, rather than from the
> UFFDIO_REGISTER anyway:
>
> uint64_t get_expected_ioctls(uint64_t mode)
> {
> uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
>
> if (test_type == TEST_HUGETLB)
> ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
>
> if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
> ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
>
> if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
> ioctls &= ~(1 << _UFFDIO_CONTINUE);
>
> return ioctls;
> }
>
> It means it'll succeed or fail depending on what kernel we run this test
> on, and also on what headers we compile the test against.
>
> I actually mentioned some of the reasoning in a follow up patch (sorry
> maybe the split here caused some confusion):
>
> selftests/mm: uffd_[un]register()
>
> Add two helpers to register/unregister to an uffd. Use them to drop
> duplicate codes.
>
> This patch also drops assert_expected_ioctls_present() and
> get_expected_ioctls(). Reasons:
>
> - It'll need a lot of effort to pass test_type==HUGETLB into it from the
> upper, so it's the simplest way to get rid of another global var
>
> - The ioctls returned in UFFDIO_REGISTER is hardly useful at all, because
> any app can already detect kernel support on any ioctl via its
> corresponding UFFD_FEATURE_*. The check here is for sanity mostly but
> it's probably destined no user app will even use it.
>
> - It's not friendly to one future goal of uffd to run on old kernels, the
> problem is get_expected_ioctls() compiles against UFFD_API_RANGE_IOCTLS,
> which is a value that can change depending on where the test is compiled,
> rather than reflecting what the kernel underneath has. It means it'll
> report false negatives on old kernels so it's against our will.
>
> So let's make our live easier.
>
> But I do agree that it's helpful to keep a test against
> uffdio_register.ioctls in this case against _UFFDIO_ZEROPAGE, so it can be
> detected dynamically. IOW, even if we would like to avoid "test !=
> HUGETLB" here, at least we should like to fix that with the UFFDIO_REGISTER
> results.
>
> Here's my offer below. :)
>
> Could I keep this patch as-is (as part of getting rid of
> get_expected_ioctls() effort; I can squash this one into "selftests/mm:
> uffd_[un]register()" if any of you think proper), meanwhile I'll squash a
> fixup to the "move zeropage test into uffd-unit-tests" explicitly check
> uffdio_register.ioctls in the same patchset? IOW, we'll have a few test
> commits missing this specific ioctl test, but then we'll have a better one
> dynamically detected from the kernel.
>
> The fixup patch attached. I think it'll automatically work when someone
> would like to introduce UFFDIO_ZEROPAGE to hugetlb too, another side
> benefit is I merged the zeropage test into one, which does look better too.
I agree that it makes sense. A nit below :)
> Thanks,
>
> --
> Peter Xu
> From 5b06f921cf8420600c697a3072a1459a5cb4956b Mon Sep 17 00:00:00 2001
> From: Peter Xu <[email protected]>
> Date: Mon, 3 Apr 2023 11:57:07 -0400
> Subject: [PATCH] fixup! selftests/mm: Move zeropage test into uffd unit tests
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-unit-tests.c | 62 +++++++++++---------
> 1 file changed, 33 insertions(+), 29 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index 793931da5056..247700bb4dd0 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -711,54 +711,58 @@ static bool do_uffdio_zeropage(int ufd, bool has_zeropage)
> return false;
> }
>
> +/*
> + * Registers a range with MISSING mode only for zeropage test. Return true
> + * if UFFDIO_ZEROPAGE supported, false otherwise. Can't use uffd_register()
> + * because we want to detect .ioctls along the way.
> + */
> +static bool
> +uffd_register_detect_zp(int uffd, void *addr, uint64_t len)
Let's spell out 'zp' as zeropage, what do you say?
> +{
> + struct uffdio_register uffdio_register = { 0 };
> + uint64_t mode = UFFDIO_REGISTER_MODE_MISSING;
> +
> + uffdio_register.range.start = (unsigned long)addr;
> + uffdio_register.range.len = len;
> + uffdio_register.mode = mode;
> +
> + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1)
> + err("zeropage test register fail");
> +
> + return uffdio_register.ioctls & (1 << _UFFDIO_ZEROPAGE);
> +}
> +
> +
> /* exercise UFFDIO_ZEROPAGE */
> -static void uffd_zeropage_test_common(bool has_zeropage)
> +static void uffd_zeropage_test(void)
> {
> - if (uffd_register(uffd, area_dst, page_size,
> - true, false, false))
> - err("register");
> + bool has_zeropage;
> + int i;
>
> + has_zeropage = uffd_register_detect_zp(uffd, area_dst, page_size);
> if (area_dst_alias)
> - if (uffd_register(uffd, area_dst_alias, page_size,
> - true, false, false))
> - err("register");
> -
> - if (do_uffdio_zeropage(uffd, has_zeropage)) {
> - int i;
> + /* Ignore the retval; we already have it */
> + uffd_register_detect_zp(uffd, area_dst_alias, page_size);
>
> + if (do_uffdio_zeropage(uffd, has_zeropage))
> for (i = 0; i < page_size; i++)
> if (area_dst[i] != 0)
> err("data non-zero at offset %d\n", i);
> - }
>
> + if (uffd_unregister(uffd, area_dst, page_size))
> + err("unregister");
>
> - if (uffd_unregister(uffd, area_dst, page_size * nr_pages))
> + if (area_dst_alias && uffd_unregister(uffd, area_dst_alias, page_size))
> err("unregister");
>
> uffd_test_pass();
> }
>
> -static void uffd_zeropage_test(void)
> -{
> - uffd_zeropage_test_common(true);
> -}
> -
> -static void uffd_zeropage_hugetlb_test(void)
> -{
> - uffd_zeropage_test_common(false);
> -}
> -
> uffd_test_case_t uffd_tests[] = {
> {
> .name = "zeropage",
> .uffd_fn = uffd_zeropage_test,
> - .mem_targets = MEM_ANON | MEM_SHMEM | MEM_SHMEM_PRIVATE,
> - .uffd_feature_required = 0,
> - },
> - {
> - .name = "zeropage-hugetlb",
> - .uffd_fn = uffd_zeropage_hugetlb_test,
> - .mem_targets = MEM_HUGETLB | MEM_HUGETLB_PRIVATE,
> + .mem_targets = MEM_ALL,
> .uffd_feature_required = 0,
> },
> {
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:17PM -0400, Peter Xu wrote:
> The idea was trying to flip this var in the alarm handler from time to time
> to test -EEXIST of UFFDIO_ZEROPAGE, but firstly it's only used in the
> zeropage test so probably only used once, meanwhile we passed
> "retry==false" so it'll never got tested anyway.
>
> Drop both sides so we always test UFFDIO_ZEROPAGE retries if has_zeropage
> is set (!hugetlb).
>
> One more thing to do is doing UFFDIO_REGISTER for the alias buffer too,
> because otherwise the test won't even pass! We were just lucky that this
> test never really got ran at all.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 20 +++++++++++---------
> 1 file changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index d724f1c78847..3487ec0bfcc8 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -88,7 +88,6 @@ static bool test_dev_userfaultfd;
> /* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */
> #define ALARM_INTERVAL_SECS 10
> static volatile bool test_uffdio_copy_eexist = true;
> -static volatile bool test_uffdio_zeropage_eexist = true;
> /* Whether to test uffd write-protection */
> static bool test_uffdio_wp = true;
> /* Whether to test uffd minor faults */
> @@ -1114,7 +1113,7 @@ static void retry_uffdio_zeropage(int ufd,
> }
> }
>
> -static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
> +static int __uffdio_zeropage(int ufd, unsigned long offset)
> {
> struct uffdio_zeropage uffdio_zeropage;
> int ret;
> @@ -1138,11 +1137,8 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
> if (res != page_size) {
> err("UFFDIO_ZEROPAGE unexpected size");
> } else {
> - if (test_uffdio_zeropage_eexist && retry) {
> - test_uffdio_zeropage_eexist = false;
> - retry_uffdio_zeropage(ufd, &uffdio_zeropage,
> - offset);
> - }
> + retry_uffdio_zeropage(ufd, &uffdio_zeropage,
> + offset);
> return 1;
> }
> } else
> @@ -1153,7 +1149,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
>
> static int uffdio_zeropage(int ufd, unsigned long offset)
> {
> - return __uffdio_zeropage(ufd, offset, false);
> + return __uffdio_zeropage(ufd, offset);
> }
>
> /* exercise UFFDIO_ZEROPAGE */
> @@ -1177,6 +1173,13 @@ static int userfaultfd_zeropage_test(void)
> assert_expected_ioctls_present(
> uffdio_register.mode, uffdio_register.ioctls);
>
> + if (area_dst_alias) {
> + /* Needed this to test zeropage-retry on shared memory */
> + uffdio_register.range.start = (unsigned long) area_dst_alias;
> + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + err("register failure");
> + }
> +
> if (uffdio_zeropage(uffd, 0))
> if (my_bcmp(area_dst, zeropage, page_size))
> err("zeropage is not zero");
> @@ -1763,7 +1766,6 @@ static void sigalrm(int sig)
> if (sig != SIGALRM)
> abort();
> test_uffdio_copy_eexist = true;
> - test_uffdio_zeropage_eexist = true;
> alarm(ALARM_INTERVAL_SECS);
> }
>
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:20PM -0400, Peter Xu wrote:
> Move common utility functions into uffd-common.[ch] files from the original
> userfaultfd.c. This prepares for a split of userfaultfd.c into two tests:
> one to only cover the old but powerful stress test, the other one covers
> all the functional tests.
>
> This movement is kind of a brute-force effort for now, with light touch-ups
> but nothing should really change. There's chances to optimize more, but
> let's leave that for later.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/Makefile | 2 +
> tools/testing/selftests/mm/uffd-common.c | 611 ++++++++++++++++++++
> tools/testing/selftests/mm/uffd-common.h | 117 ++++
> tools/testing/selftests/mm/userfaultfd.c | 694 +----------------------
> 4 files changed, 731 insertions(+), 693 deletions(-)
> create mode 100644 tools/testing/selftests/mm/uffd-common.c
> create mode 100644 tools/testing/selftests/mm/uffd-common.h
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:23PM -0400, Peter Xu wrote:
> In many ways it's weird and unwanted to keep all the tests in the same
> userfaultfd.c at least when still in the current way.
>
> For example, it doesn't make much sense to run the stress test for each
> method we can create an userfaultfd handle (either via syscall or /dev/
> node). It's a waste of time running this twice for the whole stress as the
> stress paths are the same, only the open path is different.
>
> It's also just weird to need to manually specify different types of memory
> to run all unit tests for the userfaultfd interface. We should be able to
> just run a single program and that should go through all functional uffd
> tests without running the stress test at all. The stress test was more for
> torturing and finding race conditions. We don't want to wait for stress to
> finish just to regress test a functional test.
>
> When we start to pile up more things on top of the same file and same
> functions, things start to go a bit chaos and the code is just harder to
> maintain too with tons of global variables.
>
> This patch creates a new test uffd-unit-tests to keep userfaultfd unit
> tests in the future, currently empty.
>
> Meanwhile rename the old userfaultfd.c test to uffd-stress.c.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/.gitignore | 3 ++-
> tools/testing/selftests/mm/Makefile | 8 +++---
> tools/testing/selftests/mm/run_vmtests.sh | 10 ++++---
> .../mm/{userfaultfd.c => uffd-stress.c} | 0
> tools/testing/selftests/mm/uffd-unit-tests.c | 27 +++++++++++++++++++
> 5 files changed, 40 insertions(+), 8 deletions(-)
> rename tools/testing/selftests/mm/{userfaultfd.c => uffd-stress.c} (100%)
> create mode 100644 tools/testing/selftests/mm/uffd-unit-tests.c
>
> diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
> index 347277f2adc3..8917455f4f51 100644
> --- a/tools/testing/selftests/mm/.gitignore
> +++ b/tools/testing/selftests/mm/.gitignore
> @@ -21,7 +21,8 @@ protection_keys
> protection_keys_32
> protection_keys_64
> madv_populate
> -userfaultfd
> +uffd-stress
> +uffd-unit-tests
> mlock-intersect-test
> mlock-random-test
> virtual_address_range
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 9c3737285f8a..ca8bc4ac71e6 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -20,7 +20,7 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p
> # Avoid accidental wrong builds, due to built-in rules working just a little
> # bit too well--but not quite as well as required for our situation here.
> #
> -# In other words, "make userfaultfd" is supposed to fail to build at all,
> +# In other words, "make $SOME_TEST" is supposed to fail to build at all,
> # because this Makefile only supports either "make" (all), or "make /full/path".
> # However, the built-in rules, if not suppressed, will pick up CFLAGS and the
> # initial LDLIBS (but not the target-specific LDLIBS, because those are only
> @@ -56,7 +56,8 @@ TEST_GEN_PROGS += mremap_test
> TEST_GEN_PROGS += on-fault-limit
> TEST_GEN_PROGS += thuge-gen
> TEST_GEN_PROGS += transhuge-stress
> -TEST_GEN_PROGS += userfaultfd
> +TEST_GEN_PROGS += uffd-stress
> +TEST_GEN_PROGS += uffd-unit-tests
> TEST_GEN_PROGS += soft-dirty
> TEST_GEN_PROGS += split_huge_page_test
> TEST_GEN_PROGS += ksm_tests
> @@ -107,7 +108,8 @@ include ../lib.mk
>
> $(TEST_GEN_PROGS): vm_util.c
>
> -$(OUTPUT)/userfaultfd: uffd-common.c
> +$(OUTPUT)/uffd-stress: uffd-common.c
> +$(OUTPUT)/uffd-unit-tests: uffd-common.c
>
> ifeq ($(MACHINE),x86_64)
> BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index 9cc33984aa9f..af7bbc74cd83 100644
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -196,14 +196,16 @@ CATEGORY="gup_test" run_test ./gup_test -a
> # Dump pages 0, 19, and 4096, using pin_user_pages:
> CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000
>
> +CATEGORY="userfaultfd" run_test ./uffd-unit-tests
> uffd_mods=("" ":dev")
> +uffd_stress_bin=./uffd-stress
> for mod in "${uffd_mods[@]}"; do
> - CATEGORY="userfaultfd" run_test ./userfaultfd anon${mod} 20 16
> + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16
> # Hugetlb tests require source and destination huge pages. Pass in half
> # the size ($half_ufd_size_MB), which is used for *each*.
> - CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32
> - CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32
> - CATEGORY="userfaultfd" run_test ./userfaultfd shmem${mod} 20 16
> + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32
> + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32
> + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16
> done
>
> #cleanup
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/uffd-stress.c
> similarity index 100%
> rename from tools/testing/selftests/mm/userfaultfd.c
> rename to tools/testing/selftests/mm/uffd-stress.c
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> new file mode 100644
> index 000000000000..6857388783be
> --- /dev/null
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -0,0 +1,27 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Userfaultfd unit tests.
> + *
> + * Copyright (C) 2015-2023 Red Hat, Inc.
> + */
> +
> +#include "uffd-common.h"
> +
> +#ifdef __NR_userfaultfd
> +
> +int main(int argc, char *argv[])
> +{
> + return KSFT_PASS;
> +}
> +
> +#else /* __NR_userfaultfd */
> +
> +#warning "missing __NR_userfaultfd definition"
> +
> +int main(void)
> +{
> + printf("Skipping %s (missing __NR_userfaultfd)\n", __file__);
> + return KSFT_SKIP;
> +}
> +
> +#endif /* __NR_userfaultfd */
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:49PM -0400, Peter Xu wrote:
> Provide two helpers to open an uffd handle. Drop the error checks around
> SKIPs because it's inside an errexit() anyway, which IMHO doesn't really
> help much if the test will not continue.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-common.c | 28 +++++-------------------
> tools/testing/selftests/mm/vm_util.c | 24 ++++++++++++++++++++
> 2 files changed, 29 insertions(+), 23 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index 17f2bb82c3db..3a9b5c1aca9d 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -192,34 +192,16 @@ void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
> printf("\n");
> }
>
> -static int __userfaultfd_open_dev(void)
> -{
> - int fd, _uffd;
> -
> - fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> - if (fd < 0)
> - errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
> -
> - _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
> - if (_uffd < 0)
> - errexit(errno == ENOTTY ? KSFT_SKIP : 1,
> - "creating userfaultfd failed");
> - close(fd);
> - return _uffd;
> -}
> -
> void userfaultfd_open(uint64_t *features)
> {
> struct uffdio_api uffdio_api;
>
> if (test_dev_userfaultfd)
> - uffd = __userfaultfd_open_dev();
> - else {
> - uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
> - if (uffd < 0)
> - errexit(errno == ENOSYS ? KSFT_SKIP : 1,
> - "creating userfaultfd failed");
> - }
> + uffd = uffd_open_dev(UFFD_FLAGS);
> + else
> + uffd = uffd_open_sys(UFFD_FLAGS);
> + if (uffd < 0)
> + err("uffd open failed (dev=%d)", test_dev_userfaultfd);
> uffd_flags = fcntl(uffd, F_GETFD, NULL);
>
> uffdio_api.api = UFFD_API;
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 10e76400ed70..7c2bf88d6393 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -3,6 +3,8 @@
> #include <fcntl.h>
> #include <sys/ioctl.h>
> #include <linux/userfaultfd.h>
> +#include <sys/syscall.h>
> +#include <unistd.h>
> #include "../kselftest.h"
> #include "vm_util.h"
>
> @@ -230,3 +232,25 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
>
> return ret;
> }
> +
> +int uffd_open_dev(unsigned int flags)
> +{
> + int fd, uffd;
> +
> + fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> + if (fd < 0)
> + return fd;
> + uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
> + close(fd);
> +
> + return uffd;
> +}
> +
> +int uffd_open_sys(unsigned int flags)
> +{
> +#ifdef __NR_userfaultfd
> + return syscall(__NR_userfaultfd, flags);
> +#else
> + return -1;
> +#endif
> +}
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:07:47PM -0400, Peter Xu wrote:
> Add two helpers to register/unregister to an uffd. Use them to drop
> duplicate codes.
>
> This patch also drops assert_expected_ioctls_present() and
> get_expected_ioctls(). Reasons:
>
> - It'll need a lot of effort to pass test_type==HUGETLB into it from the
> upper, so it's the simplest way to get rid of another global var
>
> - The ioctls returned in UFFDIO_REGISTER is hardly useful at all, because
> any app can already detect kernel support on any ioctl via its
> corresponding UFFD_FEATURE_*. The check here is for sanity mostly but
> it's probably destined no user app will even use it.
>
> - It's not friendly to one future goal of uffd to run on old kernels, the
> problem is get_expected_ioctls() compiles against UFFD_API_RANGE_IOCTLS,
> which is a value that can change depending on where the test is compiled,
> rather than reflecting what the kernel underneath has. It means it'll
> report false negatives on old kernels so it's against our will.
>
> So let's make our live easier.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/hugepage-mremap.c | 7 +-
> .../selftests/mm/ksm_functional_tests.c | 6 +-
> tools/testing/selftests/mm/uffd-common.c | 28 +------
> tools/testing/selftests/mm/uffd-common.h | 2 -
> tools/testing/selftests/mm/uffd-stress.c | 83 +++++--------------
> tools/testing/selftests/mm/vm_util.c | 37 +++++++++
> tools/testing/selftests/mm/vm_util.h | 7 ++
> 7 files changed, 66 insertions(+), 104 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/hugepage-mremap.c b/tools/testing/selftests/mm/hugepage-mremap.c
> index e53b5eaa8fce..2084692fe1c4 100644
> --- a/tools/testing/selftests/mm/hugepage-mremap.c
> +++ b/tools/testing/selftests/mm/hugepage-mremap.c
> @@ -60,7 +60,6 @@ static void register_region_with_uffd(char *addr, size_t len)
> {
> long uffd; /* userfaultfd file descriptor */
> struct uffdio_api uffdio_api;
> - struct uffdio_register uffdio_register;
>
> /* Create and enable userfaultfd object. */
>
> @@ -96,11 +95,7 @@ static void register_region_with_uffd(char *addr, size_t len)
> * handling by the userfaultfd object. In mode, we request to track
> * missing pages (i.e., pages that have not yet been faulted in).
> */
> -
> - uffdio_register.range.start = (unsigned long)addr;
> - uffdio_register.range.len = len;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1) {
> + if (uffd_register(uffd, addr, len, true, false, false)) {
I'd replace booleans with a bit flags as it easier to read.
Other than that LGTM.
> perror("ioctl-UFFDIO_REGISTER");
> exit(1);
> }
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:08:09PM -0400, Peter Xu wrote:
> hpage_size was wrongly used. Sometimes it means hugetlb default size,
> sometimes it was used as thp size.
>
> Remove the global variable and use the right one at each place.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-common.c | 7 ++++---
> tools/testing/selftests/mm/uffd-common.h | 2 +-
> tools/testing/selftests/mm/uffd-stress.c | 6 +++---
> 3 files changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index a2b6e4957d0f..025e40ffc7bf 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -10,7 +10,7 @@
> #define BASE_PMD_ADDR ((void *)(1UL << 30))
>
> volatile bool test_uffdio_copy_eexist = true;
> -unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
> +unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
> char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
> int uffd = -1, uffd_flags, finished, *pipefd, test_type;
> bool map_shared, test_collapse, test_dev_userfaultfd;
> @@ -115,7 +115,7 @@ static void shmem_release_pages(char *rel_area)
> static void shmem_allocate_area(void **alloc_area, bool is_src)
> {
> void *area_alias = NULL;
> - size_t bytes = nr_pages * page_size;
> + size_t bytes = nr_pages * page_size, hpage_size = read_pmd_pagesize();
> unsigned long offset = is_src ? 0 : bytes;
> char *p = NULL, *p_alias = NULL;
> int mem_fd = uffd_mem_fd_create(bytes * 2, false);
> @@ -159,7 +159,8 @@ static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
>
> static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
> {
> - if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
> + if (!check_huge_shmem(area_dst_alias, expect_nr_hpages,
> + read_pmd_pagesize()))
> err("Did not find expected %d number of hugepages",
> expect_nr_hpages);
> }
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> index 0dfab7057295..47565b2f2dee 100644
> --- a/tools/testing/selftests/mm/uffd-common.h
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -85,7 +85,7 @@ struct uffd_test_ops {
> };
> typedef struct uffd_test_ops uffd_test_ops_t;
>
> -extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
> +extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
> extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
> extern int uffd, uffd_flags, finished, *pipefd, test_type;
> extern bool map_shared, test_collapse, test_dev_userfaultfd;
> diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
> index 4eca1a0276c2..54fc9b4ffa3c 100644
> --- a/tools/testing/selftests/mm/uffd-stress.c
> +++ b/tools/testing/selftests/mm/uffd-stress.c
> @@ -655,7 +655,7 @@ static int userfaultfd_minor_test(void)
>
> uffd_test_ops->check_pmd_mapping(area_dst,
> nr_pages * page_size /
> - hpage_size);
> + read_pmd_pagesize());
> /*
> * This won't cause uffd-fault - it purely just makes sure there
> * was no corruption.
> @@ -997,7 +997,7 @@ static void parse_test_type_arg(const char *raw_type)
> err("Unsupported test: %s", raw_type);
>
> if (test_type == TEST_HUGETLB)
> - page_size = hpage_size;
> + page_size = default_huge_page_size();
> else
> page_size = sysconf(_SC_PAGE_SIZE);
>
> @@ -1035,6 +1035,7 @@ static void sigalrm(int sig)
> int main(int argc, char **argv)
> {
> size_t bytes;
> + size_t hpage_size = read_pmd_pagesize();
>
> if (argc < 4)
> usage();
> @@ -1043,7 +1044,6 @@ int main(int argc, char **argv)
> err("failed to arm SIGALRM");
> alarm(ALARM_INTERVAL_SECS);
>
> - hpage_size = default_huge_page_size();
> parse_test_type_arg(argv[1]);
> bytes = atol(argv[2]) * 1024 * 1024;
>
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:08:06PM -0400, Peter Xu wrote:
> Drop it by creating the memfd dynamically in the tests.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-common.c | 28 +++++++++++++++++++++++-
> tools/testing/selftests/mm/uffd-common.h | 2 +-
> tools/testing/selftests/mm/uffd-stress.c | 15 -------------
> 3 files changed, 28 insertions(+), 17 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index 3a9b5c1aca9d..a2b6e4957d0f 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -12,12 +12,32 @@
> volatile bool test_uffdio_copy_eexist = true;
> unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
> char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
> -int mem_fd, uffd = -1, uffd_flags, finished, *pipefd, test_type;
> +int uffd = -1, uffd_flags, finished, *pipefd, test_type;
> bool map_shared, test_collapse, test_dev_userfaultfd;
> bool test_uffdio_wp = true, test_uffdio_minor = false;
> unsigned long long *count_verify;
> uffd_test_ops_t *uffd_test_ops;
>
> +static int uffd_mem_fd_create(off_t mem_size, bool hugetlb)
> +{
> + unsigned int memfd_flags = 0;
> + int mem_fd;
> +
> + if (hugetlb)
> + memfd_flags = MFD_HUGETLB;
> + mem_fd = memfd_create("uffd-test", memfd_flags);
> + if (mem_fd < 0)
> + err("memfd_create");
> + if (ftruncate(mem_fd, mem_size))
> + err("ftruncate");
> + if (fallocate(mem_fd,
> + FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0,
> + mem_size))
> + err("fallocate");
> +
> + return mem_fd;
> +}
> +
> static void anon_release_pages(char *rel_area)
> {
> if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
> @@ -51,6 +71,7 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
> off_t offset = is_src ? 0 : size;
> void *area_alias = NULL;
> char **alloc_area_alias;
> + int mem_fd = uffd_mem_fd_create(size * 2, true);
>
> *alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
> (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> @@ -73,6 +94,8 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
> }
> if (area_alias)
> *alloc_area_alias = area_alias;
> +
> + close(mem_fd);
> }
>
> static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> @@ -95,6 +118,7 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
> size_t bytes = nr_pages * page_size;
> unsigned long offset = is_src ? 0 : bytes;
> char *p = NULL, *p_alias = NULL;
> + int mem_fd = uffd_mem_fd_create(bytes * 2, false);
>
> if (test_collapse) {
> p = BASE_PMD_ADDR;
> @@ -124,6 +148,8 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
> area_src_alias = area_alias;
> else
> area_dst_alias = area_alias;
> +
> + close(mem_fd);
> }
>
> static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> index 11f770391bd9..0dfab7057295 100644
> --- a/tools/testing/selftests/mm/uffd-common.h
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -87,7 +87,7 @@ typedef struct uffd_test_ops uffd_test_ops_t;
>
> extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
> extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
> -extern int mem_fd, uffd, uffd_flags, finished, *pipefd, test_type;
> +extern int uffd, uffd_flags, finished, *pipefd, test_type;
> extern bool map_shared, test_collapse, test_dev_userfaultfd;
> extern bool test_uffdio_wp, test_uffdio_minor;
> extern unsigned long long *count_verify;
> diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
> index e6d39a755082..4eca1a0276c2 100644
> --- a/tools/testing/selftests/mm/uffd-stress.c
> +++ b/tools/testing/selftests/mm/uffd-stress.c
> @@ -1090,21 +1090,6 @@ int main(int argc, char **argv)
> }
> nr_pages = nr_pages_per_cpu * nr_cpus;
>
> - if (test_type == TEST_SHMEM || test_type == TEST_HUGETLB) {
> - unsigned int memfd_flags = 0;
> -
> - if (test_type == TEST_HUGETLB)
> - memfd_flags = MFD_HUGETLB;
> - mem_fd = memfd_create(argv[0], memfd_flags);
> - if (mem_fd < 0)
> - err("memfd_create");
> - if (ftruncate(mem_fd, nr_pages * page_size * 2))
> - err("ftruncate");
> - if (fallocate(mem_fd,
> - FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0,
> - nr_pages * page_size * 2))
> - err("fallocate");
> - }
> printf("nr_pages: %lu, nr_pages_per_cpu: %lu\n",
> nr_pages, nr_pages_per_cpu);
> return userfaultfd_stress();
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:08:12PM -0400, Peter Xu wrote:
> Subject: selftests/mm: Let uffd_handle_page_fault() takes wp parameter
Nit: ^ take
> Make the handler optionally apply WP bit when resolving page faults for
> either missing or minor page faults. This move towards removing global
Nit: ^ moves
> test_uffdio_wp outside of the common code.
>
> For this, slightly abuse uffd_stats to keep one more parameter on whether
> we'd like to resolve page faults with WP bit set. Note that only the name
> is abused, it'll be better to be called uffd_args or similar but let's not
> bother for now.
Maybe one of the first commits in the series should have been
s/uffd_stats/uffd_args/g, but I realize that it's PITA, so I won't insist.
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-common.c | 17 +++++++++--------
> tools/testing/selftests/mm/uffd-common.h | 6 ++++--
> tools/testing/selftests/mm/uffd-stress.c | 16 ++++++++++------
> 3 files changed, 23 insertions(+), 16 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index 025e40ffc7bf..92b7e00efa8a 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -353,7 +353,7 @@ void wp_range(int ufd, __u64 start, __u64 len, bool wp)
> err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
> }
>
> -static void continue_range(int ufd, __u64 start, __u64 len)
> +static void continue_range(int ufd, __u64 start, __u64 len, bool wp)
> {
> struct uffdio_continue req;
> int ret;
> @@ -361,7 +361,7 @@ static void continue_range(int ufd, __u64 start, __u64 len)
> req.range.start = start;
> req.range.len = len;
> req.mode = 0;
> - if (test_uffdio_wp)
> + if (wp)
> req.mode |= UFFDIO_CONTINUE_MODE_WP;
>
> if (ioctl(ufd, UFFDIO_CONTINUE, &req))
> @@ -429,7 +429,8 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
> area_dst_alias));
> for (b = 0; b < page_size; ++b)
> area[b] = ~area[b];
> - continue_range(uffd, msg->arg.pagefault.address, page_size);
> + continue_range(uffd, msg->arg.pagefault.address, page_size,
> + stats->apply_wp);
> stats->minor_faults++;
> } else {
> /*
> @@ -459,7 +460,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
> offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
> offset &= ~(page_size-1);
>
> - if (copy_page(uffd, offset))
> + if (copy_page(uffd, offset, stats->apply_wp))
> stats->missing_faults++;
> }
> }
> @@ -555,7 +556,7 @@ static void wake_range(int ufd, unsigned long addr, unsigned long len)
> addr), exit(1);
> }
>
> -int __copy_page(int ufd, unsigned long offset, bool retry)
> +int __copy_page(int ufd, unsigned long offset, bool retry, bool wp)
> {
> struct uffdio_copy uffdio_copy;
>
> @@ -564,7 +565,7 @@ int __copy_page(int ufd, unsigned long offset, bool retry)
> uffdio_copy.dst = (unsigned long) area_dst + offset;
> uffdio_copy.src = (unsigned long) area_src + offset;
> uffdio_copy.len = page_size;
> - if (test_uffdio_wp)
> + if (wp)
> uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
> else
> uffdio_copy.mode = 0;
> @@ -587,8 +588,8 @@ int __copy_page(int ufd, unsigned long offset, bool retry)
> return 0;
> }
>
> -int copy_page(int ufd, unsigned long offset)
> +int copy_page(int ufd, unsigned long offset, bool wp)
> {
> - return __copy_page(ufd, offset, false);
> + return __copy_page(ufd, offset, false, wp);
> }
>
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> index 47565b2f2dee..f4bc73ce3b48 100644
> --- a/tools/testing/selftests/mm/uffd-common.h
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -72,6 +72,8 @@
> /* Userfaultfd test statistics */
> struct uffd_stats {
> int cpu;
> + /* Whether apply wr-protects when installing pages */
> + bool apply_wp;
> unsigned long missing_faults;
> unsigned long wp_faults;
> unsigned long minor_faults;
> @@ -104,8 +106,8 @@ void userfaultfd_open(uint64_t *features);
> int uffd_read_msg(int ufd, struct uffd_msg *msg);
> void wp_range(int ufd, __u64 start, __u64 len, bool wp);
> void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
> -int __copy_page(int ufd, unsigned long offset, bool retry);
> -int copy_page(int ufd, unsigned long offset);
> +int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
> +int copy_page(int ufd, unsigned long offset, bool wp);
> void *uffd_poll_thread(void *arg);
>
> #define TEST_ANON 1
> diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
> index 54fc9b4ffa3c..70cb0619354e 100644
> --- a/tools/testing/selftests/mm/uffd-stress.c
> +++ b/tools/testing/selftests/mm/uffd-stress.c
> @@ -97,6 +97,7 @@ static void uffd_stats_reset(struct uffd_stats *uffd_stats,
>
> for (i = 0; i < n_cpus; i++) {
> uffd_stats[i].cpu = i;
> + uffd_stats[i].apply_wp = test_uffdio_wp;
> uffd_stats[i].missing_faults = 0;
> uffd_stats[i].wp_faults = 0;
> uffd_stats[i].minor_faults = 0;
> @@ -156,7 +157,7 @@ static void *locking_thread(void *arg)
>
> static int copy_page_retry(int ufd, unsigned long offset)
> {
> - return __copy_page(ufd, offset, true);
> + return __copy_page(ufd, offset, true, test_uffdio_wp);
> }
>
> pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;
> @@ -309,7 +310,7 @@ static void sighndl(int sig, siginfo_t *siginfo, void *ptr)
> * This also tests UFFD_FEATURE_EVENT_FORK event along with the signal
> * feature. Using monitor thread, verify no userfault events are generated.
> */
> -static int faulting_process(int signal_test)
> +static int faulting_process(int signal_test, bool wp)
> {
> unsigned long nr;
> unsigned long long count;
> @@ -344,7 +345,7 @@ static int faulting_process(int signal_test)
> if (steps == 1) {
> /* This is a MISSING request */
> steps++;
> - if (copy_page(uffd, offset))
> + if (copy_page(uffd, offset, wp))
> signalled++;
> } else {
> /* This is a WP request */
> @@ -508,6 +509,7 @@ static int userfaultfd_events_test(void)
> true, test_uffdio_wp, false))
> err("register failure");
>
> + stats.apply_wp = test_uffdio_wp;
> if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
> err("uffd_poll_thread create");
>
> @@ -516,7 +518,7 @@ static int userfaultfd_events_test(void)
> err("fork");
>
> if (!pid)
> - exit(faulting_process(0));
> + exit(faulting_process(0, test_uffdio_wp));
>
> waitpid(pid, &err, 0);
> if (err)
> @@ -552,11 +554,12 @@ static int userfaultfd_sig_test(void)
> true, test_uffdio_wp, false))
> err("register failure");
>
> - if (faulting_process(1))
> + if (faulting_process(1, test_uffdio_wp))
> err("faulting process failed");
>
> uffd_test_ops->release_pages(area_dst);
>
> + stats.apply_wp = test_uffdio_wp;
> if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
> err("uffd_poll_thread create");
>
> @@ -565,7 +568,7 @@ static int userfaultfd_sig_test(void)
> err("fork");
>
> if (!pid)
> - exit(faulting_process(2));
> + exit(faulting_process(2, test_uffdio_wp));
>
> waitpid(pid, &err, 0);
> if (err)
> @@ -629,6 +632,7 @@ static int userfaultfd_minor_test(void)
> page_size);
> }
>
> + stats.apply_wp = test_uffdio_wp;
> if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
> err("uffd_poll_thread create");
>
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:08:15PM -0400, Peter Xu wrote:
> Mostly to detect hugetlb allocation errors and skip hugetlb tests when
> pages are not allocated.
Wouldn't we want to skip anon and shmem tests as well for consistency?
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-common.c | 32 +++++++++++++++++-------
> tools/testing/selftests/mm/uffd-common.h | 4 +--
> 2 files changed, 25 insertions(+), 11 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index 92b7e00efa8a..ae6b61144b53 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -44,10 +44,13 @@ static void anon_release_pages(char *rel_area)
> err("madvise(MADV_DONTNEED) failed");
> }
>
> -static void anon_allocate_area(void **alloc_area, bool is_src)
> +static int anon_allocate_area(void **alloc_area, bool is_src)
> {
> *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
> MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
> + if (*alloc_area == MAP_FAILED)
> + err("ENOMEM");
> + return 0;
> }
>
> static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> @@ -65,7 +68,7 @@ static void hugetlb_release_pages(char *rel_area)
> }
> }
>
> -static void hugetlb_allocate_area(void **alloc_area, bool is_src)
> +static int hugetlb_allocate_area(void **alloc_area, bool is_src)
> {
> off_t size = nr_pages * page_size;
> off_t offset = is_src ? 0 : size;
> @@ -77,14 +80,16 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
> (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> (is_src ? 0 : MAP_NORESERVE),
> mem_fd, offset);
> - if (*alloc_area == MAP_FAILED)
> - err("mmap of hugetlbfs file failed");
> + if (*alloc_area == MAP_FAILED) {
> + *alloc_area = NULL;
> + return -errno;
> + }
>
> if (map_shared) {
> area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
> MAP_SHARED, mem_fd, offset);
> if (area_alias == MAP_FAILED)
> - err("mmap of hugetlb file alias failed");
> + return -errno;
> }
>
> if (is_src) {
> @@ -96,6 +101,7 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
> *alloc_area_alias = area_alias;
>
> close(mem_fd);
> + return 0;
> }
>
> static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> @@ -112,7 +118,7 @@ static void shmem_release_pages(char *rel_area)
> err("madvise(MADV_REMOVE) failed");
> }
>
> -static void shmem_allocate_area(void **alloc_area, bool is_src)
> +static int shmem_allocate_area(void **alloc_area, bool is_src)
> {
> void *area_alias = NULL;
> size_t bytes = nr_pages * page_size, hpage_size = read_pmd_pagesize();
> @@ -150,6 +156,7 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
> area_dst_alias = area_alias;
>
> close(mem_fd);
> + return 0;
> }
>
> static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> @@ -282,14 +289,19 @@ static void uffd_test_ctx_clear(void)
> munmap_area((void **)&area_remap);
> }
>
> -void uffd_test_ctx_init(uint64_t features)
> +int uffd_test_ctx_init(uint64_t features)
> {
> unsigned long nr, cpu;
> + int ret;
>
> uffd_test_ctx_clear();
>
> - uffd_test_ops->allocate_area((void **)&area_src, true);
> - uffd_test_ops->allocate_area((void **)&area_dst, false);
> + ret = uffd_test_ops->allocate_area((void **)&area_src, true);
> + if (ret)
> + return ret;
> + ret = uffd_test_ops->allocate_area((void **)&area_dst, false);
> + if (ret)
> + return ret;
>
> userfaultfd_open(&features);
>
> @@ -337,6 +349,8 @@ void uffd_test_ctx_init(uint64_t features)
> for (cpu = 0; cpu < nr_cpus; cpu++)
> if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
> err("pipe");
> +
> + return 0;
> }
>
> void wp_range(int ufd, __u64 start, __u64 len, bool wp)
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> index f4bc73ce3b48..51ec75f6d0c1 100644
> --- a/tools/testing/selftests/mm/uffd-common.h
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -80,7 +80,7 @@ struct uffd_stats {
> };
>
> struct uffd_test_ops {
> - void (*allocate_area)(void **alloc_area, bool is_src);
> + int (*allocate_area)(void **alloc_area, bool is_src);
> void (*release_pages)(char *rel_area);
> void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
> void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
> @@ -101,7 +101,7 @@ extern uffd_test_ops_t hugetlb_uffd_test_ops;
> extern uffd_test_ops_t *uffd_test_ops;
>
> void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
> -void uffd_test_ctx_init(uint64_t features);
> +int uffd_test_ctx_init(uint64_t features);
> void userfaultfd_open(uint64_t *features);
> int uffd_read_msg(int ufd, struct uffd_msg *msg);
> void wp_range(int ufd, __u64 start, __u64 len, bool wp);
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:08:18PM -0400, Peter Xu wrote:
> Add a framework to be prepared to move unit tests from uffd-stress.c into
> uffd-unit-tests.c. The goal is to allow detection of uffd features for
> each test, and also loop over specified types of memory that a test support.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-unit-tests.c | 125 +++++++++++++++++++
> tools/testing/selftests/mm/vm_util.c | 27 ++++
> tools/testing/selftests/mm/vm_util.h | 1 +
> 3 files changed, 153 insertions(+)
>
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index dfb44ffad5f5..007145063363 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -9,6 +9,66 @@
>
> #ifdef __NR_userfaultfd
>
> +/* The unit test doesn't need a large or random size, make it 32MB for now */
> +#define UFFD_TEST_MEM_SIZE (32UL << 20)
> +
> +#define MEM_ANON BIT_ULL(0)
> +#define MEM_SHMEM BIT_ULL(1)
> +#define MEM_SHMEM_PRIVATE BIT_ULL(2)
> +#define MEM_HUGETLB BIT_ULL(3)
> +#define MEM_HUGETLB_PRIVATE BIT_ULL(4)
> +
> +struct mem_type {
> + const char *name;
> + unsigned int mem_flag;
> + uffd_test_ops_t *mem_ops;
> + bool shared;
> +};
> +typedef struct mem_type mem_type_t;
> +
> +mem_type_t mem_types[] = {
> + {
> + .name = "anon",
> + .mem_flag = MEM_ANON,
> + .mem_ops = &anon_uffd_test_ops,
> + .shared = false,
> + },
> + {
> + .name = "shmem",
> + .mem_flag = MEM_SHMEM,
> + .mem_ops = &shmem_uffd_test_ops,
> + .shared = true,
> + },
> + {
> + .name = "shmem-private",
> + .mem_flag = MEM_SHMEM_PRIVATE,
> + .mem_ops = &shmem_uffd_test_ops,
> + .shared = false,
> + },
> + {
> + .name = "hugetlb",
> + .mem_flag = MEM_HUGETLB,
> + .mem_ops = &hugetlb_uffd_test_ops,
> + .shared = true,
> + },
> + {
> + .name = "hugetlb-private",
> + .mem_flag = MEM_HUGETLB_PRIVATE,
> + .mem_ops = &hugetlb_uffd_test_ops,
> + .shared = false,
> + },
> +};
> +
> +/* Returns: UFFD_TEST_* */
> +typedef void (*uffd_test_fn)(void);
> +
> +typedef struct {
> + const char *name;
> + uffd_test_fn uffd_fn;
> + unsigned int mem_targets;
> + uint64_t uffd_feature_required;
> +} uffd_test_case_t;
> +
> struct {
> unsigned int pass, skip, fail, total;
> } uffd_test_acct;
> @@ -108,9 +168,50 @@ static int test_uffd_api(bool use_dev)
> return 1;
> }
>
> +/*
> + * This function initializes the global variables. TODO: remove global
> + * vars and then remove this.
> + */
> +static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type)
> +{
> + map_shared = mem_type->shared;
> + uffd_test_ops = mem_type->mem_ops;
> +
> + if (mem_type->mem_flag & (MEM_HUGETLB_PRIVATE | MEM_HUGETLB))
> + page_size = default_huge_page_size();
> + else
> + page_size = psize();
> +
> + nr_pages = UFFD_TEST_MEM_SIZE / page_size;
> + /* TODO: remove this global var.. it's so ugly */
> + nr_cpus = 1;
> +
> + return uffd_test_ctx_init(test->uffd_feature_required);
> +}
> +
> +static bool uffd_feature_supported(uffd_test_case_t *test)
> +{
> + uint64_t features;
> +
> + if (uffd_get_features(&features))
> + return false;
> +
> + return (features & test->uffd_feature_required) ==
> + test->uffd_feature_required;
> +}
> +
> +uffd_test_case_t uffd_tests[] = {
> +};
> +
> int main(int argc, char *argv[])
> {
> + int n_tests = sizeof(uffd_tests) / sizeof(uffd_test_case_t);
> + int n_mems = sizeof(mem_types) / sizeof(mem_type_t);
> + uffd_test_case_t *test;
> + mem_type_t *mem_type;
> + char test_name[128];
> int has_uffd;
> + int i, j;
>
> has_uffd = test_uffd_api(false);
> has_uffd |= test_uffd_api(true);
> @@ -119,7 +220,31 @@ int main(int argc, char *argv[])
> printf("Userfaultfd not supported or unprivileged, skip all tests\n");
> exit(KSFT_SKIP);
> }
> +
> + for (i = 0; i < n_tests; i++) {
> + test = &uffd_tests[i];
> + for (j = 0; j < n_mems; j++) {
> + mem_type = &mem_types[j];
> + if (!(test->mem_targets & mem_type->mem_flag))
> + continue;
> + snprintf(test_name, sizeof(test_name),
> + "%s on %s", test->name, mem_type->name);
> +
> + uffd_test_start(test_name);
> + if (!uffd_feature_supported(test)) {
> + uffd_test_skip("feature missing");
> + continue;
> + }
> + if (uffd_setup_environment(test, mem_type)) {
> + uffd_test_skip("memory allocation failed");
Maybe uffd_test_skip("environment setup failed")?
> + continue;
> + }
> + test->uffd_fn();
> + }
> + }
> +
> uffd_test_report();
> +
> return uffd_test_acct.fail ? KSFT_FAIL : KSFT_PASS;
> }
>
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 62fcf039d6b7..dad1f62a7ecd 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -264,3 +264,30 @@ int uffd_open(unsigned int flags)
>
> return uffd;
> }
> +
> +int uffd_get_features(uint64_t *features)
> +{
> + struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
> + /*
> + * This should by default work in most kernels; the feature list
> + * will be the same no matter what we pass in here.
> + */
> + int fd = uffd_open(UFFD_USER_MODE_ONLY);
> +
> + if (fd < 0)
> + /* Maybe the kernel is older than user-only mode? */
> + fd = uffd_open(0);
> +
> + if (fd < 0)
> + return fd;
> +
> + if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
> + close(fd);
> + return -errno;
> + }
> +
> + *features = uffdio_api.features;
> + close(fd);
> +
> + return 0;
> +}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index a67db8432855..2edad3256271 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -51,6 +51,7 @@ int uffd_open(unsigned int flags);
> int uffd_register(int uffd, void *addr, uint64_t len,
> bool miss, bool wp, bool minor);
> int uffd_unregister(int uffd, void *addr, uint64_t len);
> +int uffd_get_features(uint64_t *features);
>
> /*
> * On ppc64 this will only work with radix 2M hugepage size
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Thu, Mar 30, 2023 at 12:08:21PM -0400, Peter Xu wrote:
> Move it over and make it split into two tests, one for pagemap and one for
> the new WP_UNPOPULATED (to be a separate one).
>
> The thp pagemap test wasn't really working (with MADV_HUGEPAGE). Let's
> just drop it (since it never really worked anyway..) and leave that for
> later.
>
> Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Rapoport (IBM) <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-stress.c | 166 -------------------
> tools/testing/selftests/mm/uffd-unit-tests.c | 145 ++++++++++++++++
> 2 files changed, 145 insertions(+), 166 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
> index 70cb0619354e..50738a993afc 100644
> --- a/tools/testing/selftests/mm/uffd-stress.c
> +++ b/tools/testing/selftests/mm/uffd-stress.c
> @@ -671,157 +671,6 @@ static int userfaultfd_minor_test(void)
> return stats.missing_faults != 0 || stats.minor_faults != nr_pages;
> }
>
> -static int pagemap_open(void)
> -{
> - int fd = open("/proc/self/pagemap", O_RDONLY);
> -
> - if (fd < 0)
> - err("open pagemap");
> -
> - return fd;
> -}
> -
> -/* This macro let __LINE__ works in err() */
> -#define pagemap_check_wp(value, wp) do { \
> - if (!!(value & PM_UFFD_WP) != wp) \
> - err("pagemap uffd-wp bit error: 0x%"PRIx64, value); \
> - } while (0)
> -
> -static int pagemap_test_fork(bool present)
> -{
> - pid_t child = fork();
> - uint64_t value;
> - int fd, result;
> -
> - if (!child) {
> - /* Open the pagemap fd of the child itself */
> - fd = pagemap_open();
> - value = pagemap_get_entry(fd, area_dst);
> - /*
> - * After fork() uffd-wp bit should be gone as long as we're
> - * without UFFD_FEATURE_EVENT_FORK
> - */
> - pagemap_check_wp(value, false);
> - /* Succeed */
> - exit(0);
> - }
> - waitpid(child, &result, 0);
> - return result;
> -}
> -
> -static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
> -{
> - uint64_t value;
> -
> - /* Test applying pte marker to anon unpopulated */
> - wp_range(uffd, (uint64_t)area_dst, page_size, true);
> - value = pagemap_get_entry(pagemap_fd, area_dst);
> - pagemap_check_wp(value, true);
> -
> - /* Test unprotect on anon pte marker */
> - wp_range(uffd, (uint64_t)area_dst, page_size, false);
> - value = pagemap_get_entry(pagemap_fd, area_dst);
> - pagemap_check_wp(value, false);
> -
> - /* Test zap on anon marker */
> - wp_range(uffd, (uint64_t)area_dst, page_size, true);
> - if (madvise(area_dst, page_size, MADV_DONTNEED))
> - err("madvise(MADV_DONTNEED) failed");
> - value = pagemap_get_entry(pagemap_fd, area_dst);
> - pagemap_check_wp(value, false);
> -
> - /* Test fault in after marker removed */
> - *area_dst = 1;
> - value = pagemap_get_entry(pagemap_fd, area_dst);
> - pagemap_check_wp(value, false);
> - /* Drop it to make pte none again */
> - if (madvise(area_dst, page_size, MADV_DONTNEED))
> - err("madvise(MADV_DONTNEED) failed");
> -
> - /* Test read-zero-page upon pte marker */
> - wp_range(uffd, (uint64_t)area_dst, page_size, true);
> - *(volatile char *)area_dst;
> - /* Drop it to make pte none again */
> - if (madvise(area_dst, page_size, MADV_DONTNEED))
> - err("madvise(MADV_DONTNEED) failed");
> -}
> -
> -static void userfaultfd_pagemap_test(unsigned int test_pgsize)
> -{
> - int pagemap_fd;
> - uint64_t value;
> -
> - /* Pagemap tests uffd-wp only */
> - if (!test_uffdio_wp)
> - return;
> -
> - /* Not enough memory to test this page size */
> - if (test_pgsize > nr_pages * page_size)
> - return;
> -
> - printf("testing uffd-wp with pagemap (pgsize=%u): ", test_pgsize);
> - /* Flush so it doesn't flush twice in parent/child later */
> - fflush(stdout);
> -
> - uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
> -
> - if (test_pgsize > page_size) {
> - /* This is a thp test */
> - if (madvise(area_dst, nr_pages * page_size, MADV_HUGEPAGE))
> - err("madvise(MADV_HUGEPAGE) failed");
> - } else if (test_pgsize == page_size) {
> - /* This is normal page test; force no thp */
> - if (madvise(area_dst, nr_pages * page_size, MADV_NOHUGEPAGE))
> - err("madvise(MADV_NOHUGEPAGE) failed");
> - }
> -
> - if (uffd_register(uffd, area_dst, nr_pages * page_size,
> - false, true, false))
> - err("register failed");
> -
> - pagemap_fd = pagemap_open();
> -
> - /* Smoke test WP_UNPOPULATED first when it's still empty */
> - if (test_pgsize == page_size)
> - userfaultfd_wp_unpopulated_test(pagemap_fd);
> -
> - /* Touch the page */
> - *area_dst = 1;
> - wp_range(uffd, (uint64_t)area_dst, test_pgsize, true);
> - value = pagemap_get_entry(pagemap_fd, area_dst);
> - pagemap_check_wp(value, true);
> - /* Make sure uffd-wp bit dropped when fork */
> - if (pagemap_test_fork(true))
> - err("Detected stall uffd-wp bit in child");
> -
> - /* Exclusive required or PAGEOUT won't work */
> - if (!(value & PM_MMAP_EXCLUSIVE))
> - err("multiple mapping detected: 0x%"PRIx64, value);
> -
> - if (madvise(area_dst, test_pgsize, MADV_PAGEOUT))
> - err("madvise(MADV_PAGEOUT) failed");
> -
> - /* Uffd-wp should persist even swapped out */
> - value = pagemap_get_entry(pagemap_fd, area_dst);
> - pagemap_check_wp(value, true);
> - /* Make sure uffd-wp bit dropped when fork */
> - if (pagemap_test_fork(false))
> - err("Detected stall uffd-wp bit in child");
> -
> - /* Unprotect; this tests swap pte modifications */
> - wp_range(uffd, (uint64_t)area_dst, page_size, false);
> - value = pagemap_get_entry(pagemap_fd, area_dst);
> - pagemap_check_wp(value, false);
> -
> - /* Fault in the page from disk */
> - *area_dst = 2;
> - value = pagemap_get_entry(pagemap_fd, area_dst);
> - pagemap_check_wp(value, false);
> -
> - close(pagemap_fd);
> - printf("done\n");
> -}
> -
> static int userfaultfd_stress(void)
> {
> void *area;
> @@ -933,21 +782,6 @@ static int userfaultfd_stress(void)
> uffd_stats_report(uffd_stats, nr_cpus);
> }
>
> - if (test_type == TEST_ANON) {
> - /*
> - * shmem/hugetlb won't be able to run since they have different
> - * behavior on fork() (file-backed memory normally drops ptes
> - * directly when fork), meanwhile the pagemap test will verify
> - * pgtable entry of fork()ed child.
> - */
> - userfaultfd_pagemap_test(page_size);
> - /*
> - * Hard-code for x86_64 for now for 2M THP, as x86_64 is
> - * currently the only one that supports uffd-wp
> - */
> - userfaultfd_pagemap_test(page_size * 512);
> - }
> -
> return userfaultfd_zeropage_test() || userfaultfd_sig_test()
> || userfaultfd_events_test() || userfaultfd_minor_test();
> }
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index 007145063363..bcd67bd4ec90 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -200,7 +200,152 @@ static bool uffd_feature_supported(uffd_test_case_t *test)
> test->uffd_feature_required;
> }
>
> +static int pagemap_open(void)
> +{
> + int fd = open("/proc/self/pagemap", O_RDONLY);
> +
> + if (fd < 0)
> + err("open pagemap");
> +
> + return fd;
> +}
> +
> +/* This macro let __LINE__ works in err() */
> +#define pagemap_check_wp(value, wp) do { \
> + if (!!(value & PM_UFFD_WP) != wp) \
> + err("pagemap uffd-wp bit error: 0x%"PRIx64, value); \
> + } while (0)
> +
> +static int pagemap_test_fork(bool present)
> +{
> + pid_t child = fork();
> + uint64_t value;
> + int fd, result;
> +
> + if (!child) {
> + /* Open the pagemap fd of the child itself */
> + fd = pagemap_open();
> + value = pagemap_get_entry(fd, area_dst);
> + /*
> + * After fork() uffd-wp bit should be gone as long as we're
> + * without UFFD_FEATURE_EVENT_FORK
> + */
> + pagemap_check_wp(value, false);
> + /* Succeed */
> + exit(0);
> + }
> + waitpid(child, &result, 0);
> + return result;
> +}
> +
> +static void uffd_wp_unpopulated_test(void)
> +{
> + uint64_t value;
> + int pagemap_fd;
> +
> + if (uffd_register(uffd, area_dst, nr_pages * page_size,
> + false, true, false))
> + err("register failed");
> +
> + pagemap_fd = pagemap_open();
> +
> + /* Test applying pte marker to anon unpopulated */
> + wp_range(uffd, (uint64_t)area_dst, page_size, true);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> + pagemap_check_wp(value, true);
> +
> + /* Test unprotect on anon pte marker */
> + wp_range(uffd, (uint64_t)area_dst, page_size, false);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> + pagemap_check_wp(value, false);
> +
> + /* Test zap on anon marker */
> + wp_range(uffd, (uint64_t)area_dst, page_size, true);
> + if (madvise(area_dst, page_size, MADV_DONTNEED))
> + err("madvise(MADV_DONTNEED) failed");
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> + pagemap_check_wp(value, false);
> +
> + /* Test fault in after marker removed */
> + *area_dst = 1;
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> + pagemap_check_wp(value, false);
> + /* Drop it to make pte none again */
> + if (madvise(area_dst, page_size, MADV_DONTNEED))
> + err("madvise(MADV_DONTNEED) failed");
> +
> + /* Test read-zero-page upon pte marker */
> + wp_range(uffd, (uint64_t)area_dst, page_size, true);
> + *(volatile char *)area_dst;
> + /* Drop it to make pte none again */
> + if (madvise(area_dst, page_size, MADV_DONTNEED))
> + err("madvise(MADV_DONTNEED) failed");
> +
> + uffd_test_pass();
> +}
> +
> +static void uffd_pagemap_test(void)
> +{
> + int pagemap_fd;
> + uint64_t value;
> +
> + if (uffd_register(uffd, area_dst, nr_pages * page_size,
> + false, true, false))
> + err("register failed");
> +
> + pagemap_fd = pagemap_open();
> +
> + /* Touch the page */
> + *area_dst = 1;
> + wp_range(uffd, (uint64_t)area_dst, page_size, true);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> + pagemap_check_wp(value, true);
> + /* Make sure uffd-wp bit dropped when fork */
> + if (pagemap_test_fork(true))
> + err("Detected stall uffd-wp bit in child");
> +
> + /* Exclusive required or PAGEOUT won't work */
> + if (!(value & PM_MMAP_EXCLUSIVE))
> + err("multiple mapping detected: 0x%"PRIx64, value);
> +
> + if (madvise(area_dst, page_size, MADV_PAGEOUT))
> + err("madvise(MADV_PAGEOUT) failed");
> +
> + /* Uffd-wp should persist even swapped out */
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> + pagemap_check_wp(value, true);
> + /* Make sure uffd-wp bit dropped when fork */
> + if (pagemap_test_fork(false))
> + err("Detected stall uffd-wp bit in child");
> +
> + /* Unprotect; this tests swap pte modifications */
> + wp_range(uffd, (uint64_t)area_dst, page_size, false);
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> + pagemap_check_wp(value, false);
> +
> + /* Fault in the page from disk */
> + *area_dst = 2;
> + value = pagemap_get_entry(pagemap_fd, area_dst);
> + pagemap_check_wp(value, false);
> +
> + close(pagemap_fd);
> + uffd_test_pass();
> +}
> +
> uffd_test_case_t uffd_tests[] = {
> + {
> + .name = "pagemap",
> + .uffd_fn = uffd_pagemap_test,
> + .mem_targets = MEM_ANON,
> + .uffd_feature_required = UFFD_FEATURE_PAGEFAULT_FLAG_WP,
> + },
> + {
> + .name = "wp-unpopulated",
> + .uffd_fn = uffd_wp_unpopulated_test,
> + .mem_targets = MEM_ANON,
> + .uffd_feature_required =
> + UFFD_FEATURE_PAGEFAULT_FLAG_WP | UFFD_FEATURE_WP_UNPOPULATED,
> + },
> };
>
> int main(int argc, char *argv[])
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
On Fri, Apr 07, 2023 at 12:42:29PM +0300, Mike Rapoport wrote:
> > +/*
> > + * Registers a range with MISSING mode only for zeropage test. Return true
> > + * if UFFDIO_ZEROPAGE supported, false otherwise. Can't use uffd_register()
> > + * because we want to detect .ioctls along the way.
> > + */
> > +static bool
> > +uffd_register_detect_zp(int uffd, void *addr, uint64_t len)
>
> Let's spell out 'zp' as zeropage, what do you say?
Definitely can do. :)
Thanks,
--
Peter Xu
On Fri, Apr 07, 2023 at 02:08:54PM +0300, Mike Rapoport wrote:
> > @@ -96,11 +95,7 @@ static void register_region_with_uffd(char *addr, size_t len)
> > * handling by the userfaultfd object. In mode, we request to track
> > * missing pages (i.e., pages that have not yet been faulted in).
> > */
> > -
> > - uffdio_register.range.start = (unsigned long)addr;
> > - uffdio_register.range.len = len;
> > - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> > - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1) {
> > + if (uffd_register(uffd, addr, len, true, false, false)) {
>
> I'd replace booleans with a bit flags as it easier to read.
> Other than that LGTM.
It was mostly for no need to remember the long names of macros, and easier
when conditionally set with some modes. E.g., we have 5 callers have things
like:
uffd_register(..., test_uffdio_wp ? UFFDIO_REGISTER_MODE_WP : 0);
The bools simplifes it to:
uffd_register(..., test_uffdio_wp, ...);
But let me know if you still think that's better - I can switch here.
Thanks,
--
Peter Xu
On Tue, Apr 11, 2023 at 01:52:58PM +0300, Mike Rapoport wrote:
> On Thu, Mar 30, 2023 at 12:08:12PM -0400, Peter Xu wrote:
> > Subject: selftests/mm: Let uffd_handle_page_fault() takes wp parameter
>
> Nit: ^ take
>
> > Make the handler optionally apply WP bit when resolving page faults for
> > either missing or minor page faults. This move towards removing global
>
> Nit: ^ moves
Will fix.
>
> > test_uffdio_wp outside of the common code.
> >
> > For this, slightly abuse uffd_stats to keep one more parameter on whether
> > we'd like to resolve page faults with WP bit set. Note that only the name
> > is abused, it'll be better to be called uffd_args or similar but let's not
> > bother for now.
>
> Maybe one of the first commits in the series should have been
> s/uffd_stats/uffd_args/g, but I realize that it's PITA, so I won't insist.
It's fine, I'll add one patch. The rebase does take some more minutes, but
it's manageable. I'll drop this paragraph then.
Thanks,
--
Peter Xu
On Tue, Apr 11, 2023 at 02:02:30PM +0300, Mike Rapoport wrote:
> On Thu, Mar 30, 2023 at 12:08:15PM -0400, Peter Xu wrote:
> > Mostly to detect hugetlb allocation errors and skip hugetlb tests when
> > pages are not allocated.
>
> Wouldn't we want to skip anon and shmem tests as well for consistency?
Much less possibile (and useful) than hugetlb for sure, but it's indeed
more consistent. Will do.
--
Peter Xu
On Tue, Apr 11, 2023 at 02:09:51PM +0300, Mike Rapoport wrote:
> > @@ -119,7 +220,31 @@ int main(int argc, char *argv[])
> > printf("Userfaultfd not supported or unprivileged, skip all tests\n");
> > exit(KSFT_SKIP);
> > }
> > +
> > + for (i = 0; i < n_tests; i++) {
> > + test = &uffd_tests[i];
> > + for (j = 0; j < n_mems; j++) {
> > + mem_type = &mem_types[j];
> > + if (!(test->mem_targets & mem_type->mem_flag))
> > + continue;
> > + snprintf(test_name, sizeof(test_name),
> > + "%s on %s", test->name, mem_type->name);
> > +
> > + uffd_test_start(test_name);
> > + if (!uffd_feature_supported(test)) {
> > + uffd_test_skip("feature missing");
> > + continue;
> > + }
> > + if (uffd_setup_environment(test, mem_type)) {
> > + uffd_test_skip("memory allocation failed");
>
> Maybe uffd_test_skip("environment setup failed")?
Sure.
Side note: the line will be changed after "selftests/mm: Allow uffd test to
skip properly with no privilege" to contain an errmsg pointer instead, so
it won't affect the last result after series applied.
Thanks,
--
Peter Xu
On Tue, Apr 11, 2023 at 03:13:18PM -0400, Peter Xu wrote:
> On Fri, Apr 07, 2023 at 02:08:54PM +0300, Mike Rapoport wrote:
> > > @@ -96,11 +95,7 @@ static void register_region_with_uffd(char *addr, size_t len)
> > > * handling by the userfaultfd object. In mode, we request to track
> > > * missing pages (i.e., pages that have not yet been faulted in).
> > > */
> > > -
> > > - uffdio_register.range.start = (unsigned long)addr;
> > > - uffdio_register.range.len = len;
> > > - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> > > - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1) {
> > > + if (uffd_register(uffd, addr, len, true, false, false)) {
> >
> > I'd replace booleans with a bit flags as it easier to read.
> > Other than that LGTM.
>
> It was mostly for no need to remember the long names of macros, and easier
> when conditionally set with some modes. E.g., we have 5 callers have things
> like:
>
> uffd_register(..., test_uffdio_wp ? UFFDIO_REGISTER_MODE_WP : 0);
>
> The bools simplifes it to:
>
> uffd_register(..., test_uffdio_wp, ...);
>
> But let me know if you still think that's better - I can switch here.
No strong feelings, however you prefer.
> Thanks,
>
> --
> Peter Xu
>
--
Sincerely yours,
Mike.