This series is based on torvalds/master.
The series is split up like so:
- Patch 1 is a simple fixup which we should take in any case (even by itself).
- Patches 2-6 add the feature, configurable selftest support, and docs.
Why not ...?
============
- Why not /proc/[pid]/userfaultfd? The proposed use case for this is for one
process to open a userfaultfd which can intercept another process' page
faults. This seems to me like exactly what CAP_SYS_PTRACE is for, though, so I
think this use case can simply use a syscall without the powers CAP_SYS_PTRACE
grants being "too much".
- Why not use a syscall? Access to syscalls is generally controlled by
capabilities. We don't have a capability which is used for userfaultfd access
without also granting more / other permissions as well, and adding a new
capability was rejected [1].
- It's possible a LSM could be used to control access instead. I suspect
adding a brand new one just for this would be rejected, but I think some
existing ones like SELinux can be used to filter syscall access. Enabling
SELinux for large production deployments which don't already use it is
likely to be a huge undertaking though, and I don't think this use case by
itself is enough to motivate that kind of architectural change.
Changelog
=========
v3->v4:
- Picked up an Acked-by on 5/5.
- Updated cover letter to cover "why not ...".
- Refactored userfaultfd_allowed() into userfaultfd_syscall_allowed(). [Peter]
- Removed obsolete comment from a previous version. [Peter]
- Refactored userfaultfd_open() in selftest. [Peter]
- Reworded admin-guide documentation. [Mike, Peter]
- Squashed 2 commits adding /dev/userfaultfd to selftest and making selftest
configurable. [Peter]
- Added "syscall" test modifier (the default behavior) to selftest. [Peter]
v2->v3:
- Rebased onto linux-next/akpm-base, in order to be based on top of the
run_vmtests.sh refactor which was merged previously.
- Picked up some Reviewed-by's.
- Fixed ioctl definition (_IO instead of _IOWR), and stopped using
compat_ptr_ioctl since it is unneeded for ioctls which don't take a pointer.
- Removed the "handle_kernel_faults" bool, simplifying the code. The result is
logically equivalent, but simpler.
- Fixed userfaultfd selftest so it returns KSFT_SKIP appropriately.
- Reworded documentation per Shuah's feedback on v2.
- Improved example usage for userfaultfd selftest.
v1->v2:
- Add documentation update.
- Test *both* userfaultfd(2) and /dev/userfaultfd via the selftest.
[1]: https://lore.kernel.org/lkml/[email protected]/T/
Axel Rasmussen (5):
selftests: vm: add hugetlb_shared userfaultfd test to run_vmtests.sh
userfaultfd: add /dev/userfaultfd for fine grained access control
userfaultfd: selftests: modify selftest to use /dev/userfaultfd
userfaultfd: update documentation to describe /dev/userfaultfd
selftests: vm: add /dev/userfaultfd test cases to run_vmtests.sh
Documentation/admin-guide/mm/userfaultfd.rst | 41 +++++++++++-
Documentation/admin-guide/sysctl/vm.rst | 3 +
fs/userfaultfd.c | 69 ++++++++++++++++----
include/uapi/linux/userfaultfd.h | 4 ++
tools/testing/selftests/vm/run_vmtests.sh | 11 +++-
tools/testing/selftests/vm/userfaultfd.c | 69 +++++++++++++++++---
6 files changed, 169 insertions(+), 28 deletions(-)
--
2.37.0.170.g444d1eabd0-goog
This not being included was just a simple oversight. There are certain
features (like minor fault support) which are only enabled on shared
mappings, so without including hugetlb_shared we actually lose a
significant amount of test coverage.
Reviewed-by: Shuah Khan <[email protected]>
Reviewed-by: Peter Xu <[email protected]>
Signed-off-by: Axel Rasmussen <[email protected]>
---
tools/testing/selftests/vm/run_vmtests.sh | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/vm/run_vmtests.sh b/tools/testing/selftests/vm/run_vmtests.sh
index 41fce8bea929..e70ae0f3aaf6 100755
--- a/tools/testing/selftests/vm/run_vmtests.sh
+++ b/tools/testing/selftests/vm/run_vmtests.sh
@@ -121,9 +121,11 @@ run_test ./gup_test -a
run_test ./gup_test -ct -F 0x1 0 19 0x1000
run_test ./userfaultfd anon 20 16
-# Test requires source and destination huge pages. Size of source
-# (half_ufd_size_MB) is passed as argument to test.
+# Hugetlb tests require source and destination huge pages. Pass in half the
+# size ($half_ufd_size_MB), which is used for *each*.
run_test ./userfaultfd hugetlb "$half_ufd_size_MB" 32
+run_test ./userfaultfd hugetlb_shared "$half_ufd_size_MB" 32 "$mnt"/uffd-test
+rm -f "$mnt"/uffd-test
run_test ./userfaultfd shmem 20 16
#cleanup
--
2.37.0.170.g444d1eabd0-goog
Explain the different ways to create a new userfaultfd, and how access
control works for each way.
Signed-off-by: Axel Rasmussen <[email protected]>
---
Documentation/admin-guide/mm/userfaultfd.rst | 41 ++++++++++++++++++--
Documentation/admin-guide/sysctl/vm.rst | 3 ++
2 files changed, 41 insertions(+), 3 deletions(-)
diff --git a/Documentation/admin-guide/mm/userfaultfd.rst b/Documentation/admin-guide/mm/userfaultfd.rst
index 6528036093e1..a76c9dc1865b 100644
--- a/Documentation/admin-guide/mm/userfaultfd.rst
+++ b/Documentation/admin-guide/mm/userfaultfd.rst
@@ -17,7 +17,10 @@ of the ``PROT_NONE+SIGSEGV`` trick.
Design
======
-Userfaults are delivered and resolved through the ``userfaultfd`` syscall.
+Userspace creates a new userfaultfd, initializes it, and registers one or more
+regions of virtual memory with it. Then, any page faults which occur within the
+region(s) result in a message being delivered to the userfaultfd, notifying
+userspace of the fault.
The ``userfaultfd`` (aside from registering and unregistering virtual
memory ranges) provides two primary functionalities:
@@ -34,12 +37,11 @@ The real advantage of userfaults if compared to regular virtual memory
management of mremap/mprotect is that the userfaults in all their
operations never involve heavyweight structures like vmas (in fact the
``userfaultfd`` runtime load never takes the mmap_lock for writing).
-
Vmas are not suitable for page- (or hugepage) granular fault tracking
when dealing with virtual address spaces that could span
Terabytes. Too many vmas would be needed for that.
-The ``userfaultfd`` once opened by invoking the syscall, can also be
+The ``userfaultfd``, once created, can also be
passed using unix domain sockets to a manager process, so the same
manager process could handle the userfaults of a multitude of
different processes without them being aware about what is going on
@@ -50,6 +52,39 @@ is a corner case that would currently return ``-EBUSY``).
API
===
+Creating a userfaultfd
+----------------------
+
+There are two ways to create a new userfaultfd, each of which provide ways to
+restrict access to this functionality (since historically userfaultfds which
+handle kernel page faults have been a useful tool for exploiting the kernel).
+
+The first way, supported since userfaultfd was introduced, is the
+userfaultfd(2) syscall. Access to this is controlled in several ways:
+
+- Any user can always create a userfaultfd which traps userspace page faults
+ only. Such a userfaultfd can be created using the userfaultfd(2) syscall
+ with the flag UFFD_USER_MODE_ONLY.
+
+- In order to also trap kernel page faults for the address space, then either
+ the process needs the CAP_SYS_PTRACE capability, or the system must have
+ vm.unprivileged_userfaultfd set to 1. By default, vm.unprivileged_userfaultfd
+ is set to 0.
+
+The second way, added to the kernel more recently, is by opening and issuing a
+USERFAULTFD_IOC_NEW ioctl to /dev/userfaultfd. This method yields equivalent
+userfaultfds to the userfaultfd(2) syscall.
+
+Unlike userfaultfd(2), access to /dev/userfaultfd is controlled via normal
+filesystem permissions (user/group/mode), which gives fine grained access to
+userfaultfd specifically, without also granting other unrelated privileges at
+the same time (as e.g. granting CAP_SYS_PTRACE would do). Users who have access
+to /dev/userfaultfd can always create userfaultfds that trap kernel page faults;
+vm.unprivileged_userfaultfd is not considered.
+
+Initializing a userfaultfd
+--------------------------
+
When first opened the ``userfaultfd`` must be enabled invoking the
``UFFDIO_API`` ioctl specifying a ``uffdio_api.api`` value set to ``UFFD_API`` (or
a later API version) which will specify the ``read/POLLIN`` protocol
diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst
index 5c9aa171a0d3..36cf21f3b7ab 100644
--- a/Documentation/admin-guide/sysctl/vm.rst
+++ b/Documentation/admin-guide/sysctl/vm.rst
@@ -928,6 +928,9 @@ calls without any restrictions.
The default value is 0.
+Another way to control permissions for userfaultfd is to use
+/dev/userfaultfd instead of userfaultfd(2). See
+Documentation/admin-guide/mm/userfaultfd.rst.
user_reserve_kbytes
===================
--
2.37.0.170.g444d1eabd0-goog
We clearly want to ensure both userfaultfd(2) and /dev/userfaultfd keep
working into the future, so just run the test twice, using each
interface.
Instead of always testing both userfaultfd(2) and /dev/userfaultfd,
let the user choose which to test.
As with other test features, change the behavior based on a new
command line flag. Introduce the idea of "test mods", which are
generic (not specific to a test type) modifications to the behavior of
the test. This is sort of borrowed from this RFC patch series [1], but
simplified a bit.
The benefit is, in "typical" configurations this test is somewhat slow
(say, 30sec or something). Testing both clearly doubles it, so it may
not always be desirable, as users are likely to use one or the other,
but never both, in the "real world".
[1]: https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/
Signed-off-by: Axel Rasmussen <[email protected]>
---
tools/testing/selftests/vm/userfaultfd.c | 69 ++++++++++++++++++++----
1 file changed, 60 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c
index 0bdfc1955229..0a126c620bc0 100644
--- a/tools/testing/selftests/vm/userfaultfd.c
+++ b/tools/testing/selftests/vm/userfaultfd.c
@@ -77,6 +77,11 @@ static int bounces;
#define TEST_SHMEM 3
static int test_type;
+#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)
+
+/* test using /dev/userfaultfd, instead of userfaultfd(2) */
+static bool test_dev_userfaultfd;
+
/* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */
#define ALARM_INTERVAL_SECS 10
static volatile bool test_uffdio_copy_eexist = true;
@@ -125,6 +130,8 @@ struct uffd_stats {
const char *examples =
"# Run anonymous memory test on 100MiB region with 99999 bounces:\n"
"./userfaultfd anon 100 99999\n\n"
+ "# Run the same anonymous memory test, but using /dev/userfaultfd:\n"
+ "./userfaultfd anon:dev 100 99999\n\n"
"# Run share memory test on 1GiB region with 99 bounces:\n"
"./userfaultfd shmem 1000 99\n\n"
"# Run hugetlb memory test on 256MiB region with 50 bounces:\n"
@@ -141,6 +148,14 @@ static void usage(void)
"[hugetlbfs_file]\n\n");
fprintf(stderr, "Supported <test type>: anon, hugetlb, "
"hugetlb_shared, shmem\n\n");
+ fprintf(stderr, "'Test mods' can be joined to the test type string with a ':'. "
+ "Supported mods:\n");
+ fprintf(stderr, "\tsyscall - Use userfaultfd(2) (default)\n");
+ fprintf(stderr, "\tdev - Use /dev/userfaultfd instead of userfaultfd(2)\n");
+ fprintf(stderr, "\nExample test mod usage:\n");
+ fprintf(stderr, "# Run anonymous memory test with /dev/userfaultfd:\n");
+ fprintf(stderr, "./userfaultfd anon:dev 100 99999\n\n");
+
fprintf(stderr, "Examples:\n\n");
fprintf(stderr, "%s", examples);
exit(1);
@@ -154,12 +169,14 @@ static void usage(void)
ret, __LINE__); \
} while (0)
-#define err(fmt, ...) \
+#define errexit(exitcode, fmt, ...) \
do { \
_err(fmt, ##__VA_ARGS__); \
- exit(1); \
+ exit(exitcode); \
} while (0)
+#define err(fmt, ...) errexit(1, fmt, ##__VA_ARGS__)
+
static void uffd_stats_reset(struct uffd_stats *uffd_stats,
unsigned long n_cpus)
{
@@ -383,13 +400,29 @@ static void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
}
}
+static int __userfaultfd_open_dev(void)
+{
+ int fd, _uffd = -1;
+
+ fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+ if (fd < 0)
+ return -1;
+
+ _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
+ close(fd);
+ return _uffd;
+}
+
static void userfaultfd_open(uint64_t *features)
{
struct uffdio_api uffdio_api;
- uffd = syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY);
+ if (test_dev_userfaultfd)
+ uffd = __userfaultfd_open_dev();
+ else
+ uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
if (uffd < 0)
- err("userfaultfd syscall not available in this kernel");
+ errexit(KSFT_SKIP, "creating userfaultfd failed");
uffd_flags = fcntl(uffd, F_GETFD, NULL);
uffdio_api.api = UFFD_API;
@@ -1584,8 +1617,6 @@ unsigned long default_huge_page_size(void)
static void set_test_type(const char *type)
{
- uint64_t features = UFFD_API_FEATURES;
-
if (!strcmp(type, "anon")) {
test_type = TEST_ANON;
uffd_test_ops = &anon_uffd_test_ops;
@@ -1603,9 +1634,29 @@ static void set_test_type(const char *type)
test_type = TEST_SHMEM;
uffd_test_ops = &shmem_uffd_test_ops;
test_uffdio_minor = true;
- } else {
- err("Unknown test type: %s", type);
}
+}
+
+static void parse_test_type_arg(const char *raw_type)
+{
+ char *buf = strdup(raw_type);
+ uint64_t features = UFFD_API_FEATURES;
+
+ while (buf) {
+ const char *token = strsep(&buf, ":");
+
+ if (!test_type)
+ set_test_type(token);
+ else if (!strcmp(token, "dev"))
+ test_dev_userfaultfd = true;
+ else if (!strcmp(token, "syscall"))
+ test_dev_userfaultfd = false;
+ else
+ err("unrecognized test mod '%s'", token);
+ }
+
+ if (!test_type)
+ err("failed to parse test type argument: '%s'", raw_type);
if (test_type == TEST_HUGETLB)
page_size = default_huge_page_size();
@@ -1653,7 +1704,7 @@ int main(int argc, char **argv)
err("failed to arm SIGALRM");
alarm(ALARM_INTERVAL_SECS);
- set_test_type(argv[1]);
+ parse_test_type_arg(argv[1]);
nr_cpus = sysconf(_SC_NPROCESSORS_ONLN);
nr_pages_per_cpu = atol(argv[2]) * 1024*1024 / page_size /
--
2.37.0.170.g444d1eabd0-goog
This new mode was recently added to the userfaultfd selftest. We want to
exercise both userfaultfd(2) as well as /dev/userfaultfd, so add both
test cases to the script.
Reviewed-by: Shuah Khan <[email protected]>
Acked-by: Peter Xu <[email protected]>
Signed-off-by: Axel Rasmussen <[email protected]>
---
tools/testing/selftests/vm/run_vmtests.sh | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tools/testing/selftests/vm/run_vmtests.sh b/tools/testing/selftests/vm/run_vmtests.sh
index e70ae0f3aaf6..156f864030fc 100755
--- a/tools/testing/selftests/vm/run_vmtests.sh
+++ b/tools/testing/selftests/vm/run_vmtests.sh
@@ -121,12 +121,17 @@ run_test ./gup_test -a
run_test ./gup_test -ct -F 0x1 0 19 0x1000
run_test ./userfaultfd anon 20 16
+run_test ./userfaultfd anon:dev 20 16
# Hugetlb tests require source and destination huge pages. Pass in half the
# size ($half_ufd_size_MB), which is used for *each*.
run_test ./userfaultfd hugetlb "$half_ufd_size_MB" 32
+run_test ./userfaultfd hugetlb:dev "$half_ufd_size_MB" 32
run_test ./userfaultfd hugetlb_shared "$half_ufd_size_MB" 32 "$mnt"/uffd-test
rm -f "$mnt"/uffd-test
+run_test ./userfaultfd hugetlb_shared:dev "$half_ufd_size_MB" 32 "$mnt"/uffd-test
+rm -f "$mnt"/uffd-test
run_test ./userfaultfd shmem 20 16
+run_test ./userfaultfd shmem:dev 20 16
#cleanup
umount "$mnt"
--
2.37.0.170.g444d1eabd0-goog
Historically, it has been shown that intercepting kernel faults with
userfaultfd (thereby forcing the kernel to wait for an arbitrary amount
of time) can be exploited, or at least can make some kinds of exploits
easier. So, in 37cd0575b8 "userfaultfd: add UFFD_USER_MODE_ONLY" we
changed things so, in order for kernel faults to be handled by
userfaultfd, either the process needs CAP_SYS_PTRACE, or this sysctl
must be configured so that any unprivileged user can do it.
In a typical implementation of a hypervisor with live migration (take
QEMU/KVM as one such example), we do indeed need to be able to handle
kernel faults. But, both options above are less than ideal:
- Toggling the sysctl increases attack surface by allowing any
unprivileged user to do it.
- Granting the live migration process CAP_SYS_PTRACE gives it this
ability, but *also* the ability to "observe and control the
execution of another process [...], and examine and change [its]
memory and registers" (from ptrace(2)). This isn't something we need
or want to be able to do, so granting this permission violates the
"principle of least privilege".
This is all a long winded way to say: we want a more fine-grained way to
grant access to userfaultfd, without granting other additional
permissions at the same time.
To achieve this, add a /dev/userfaultfd misc device. This device
provides an alternative to the userfaultfd(2) syscall for the creation
of new userfaultfds. The idea is, any userfaultfds created this way will
be able to handle kernel faults, without the caller having any special
capabilities. Access to this mechanism is instead restricted using e.g.
standard filesystem permissions.
Signed-off-by: Axel Rasmussen <[email protected]>
---
fs/userfaultfd.c | 69 +++++++++++++++++++++++++-------
include/uapi/linux/userfaultfd.h | 4 ++
2 files changed, 59 insertions(+), 14 deletions(-)
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index e943370107d0..968f2517a281 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -30,6 +30,7 @@
#include <linux/security.h>
#include <linux/hugetlb.h>
#include <linux/swapops.h>
+#include <linux/miscdevice.h>
int sysctl_unprivileged_userfaultfd __read_mostly;
@@ -413,13 +414,8 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
if (ctx->features & UFFD_FEATURE_SIGBUS)
goto out;
- if ((vmf->flags & FAULT_FLAG_USER) == 0 &&
- ctx->flags & UFFD_USER_MODE_ONLY) {
- printk_once(KERN_WARNING "uffd: Set unprivileged_userfaultfd "
- "sysctl knob to 1 if kernel faults must be handled "
- "without obtaining CAP_SYS_PTRACE capability\n");
+ if (!(vmf->flags & FAULT_FLAG_USER) && (ctx->flags & UFFD_USER_MODE_ONLY))
goto out;
- }
/*
* If it's already released don't get it. This avoids to loop
@@ -2052,19 +2048,30 @@ static void init_once_userfaultfd_ctx(void *mem)
seqcount_spinlock_init(&ctx->refile_seq, &ctx->fault_pending_wqh.lock);
}
-SYSCALL_DEFINE1(userfaultfd, int, flags)
+static inline bool userfaultfd_syscall_allowed(int flags)
+{
+ /* Userspace-only page faults are always allowed */
+ if (flags & UFFD_USER_MODE_ONLY)
+ return true;
+
+ /*
+ * The user is requesting a userfaultfd which can handle kernel faults.
+ * Privileged users are always allowed to do this.
+ */
+ if (capable(CAP_SYS_PTRACE))
+ return true;
+
+ /* Otherwise, access to kernel fault handling is sysctl controlled. */
+ return sysctl_unprivileged_userfaultfd;
+}
+
+static int new_userfaultfd(bool is_syscall, int flags)
{
struct userfaultfd_ctx *ctx;
int fd;
- if (!sysctl_unprivileged_userfaultfd &&
- (flags & UFFD_USER_MODE_ONLY) == 0 &&
- !capable(CAP_SYS_PTRACE)) {
- printk_once(KERN_WARNING "uffd: Set unprivileged_userfaultfd "
- "sysctl knob to 1 if kernel faults must be handled "
- "without obtaining CAP_SYS_PTRACE capability\n");
+ if (is_syscall && !userfaultfd_syscall_allowed(flags))
return -EPERM;
- }
BUG_ON(!current->mm);
@@ -2098,8 +2105,42 @@ SYSCALL_DEFINE1(userfaultfd, int, flags)
return fd;
}
+SYSCALL_DEFINE1(userfaultfd, int, flags)
+{
+ return new_userfaultfd(true, flags);
+}
+
+static int userfaultfd_dev_open(struct inode *inode, struct file *file)
+{
+ return 0;
+}
+
+static long userfaultfd_dev_ioctl(struct file *file, unsigned int cmd, unsigned long flags)
+{
+ if (cmd != USERFAULTFD_IOC_NEW)
+ return -EINVAL;
+
+ return new_userfaultfd(false, flags);
+}
+
+static const struct file_operations userfaultfd_dev_fops = {
+ .open = userfaultfd_dev_open,
+ .unlocked_ioctl = userfaultfd_dev_ioctl,
+ .compat_ioctl = userfaultfd_dev_ioctl,
+ .owner = THIS_MODULE,
+ .llseek = noop_llseek,
+};
+
+static struct miscdevice userfaultfd_misc = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "userfaultfd",
+ .fops = &userfaultfd_dev_fops
+};
+
static int __init userfaultfd_init(void)
{
+ WARN_ON(misc_register(&userfaultfd_misc));
+
userfaultfd_ctx_cachep = kmem_cache_create("userfaultfd_ctx_cache",
sizeof(struct userfaultfd_ctx),
0,
diff --git a/include/uapi/linux/userfaultfd.h b/include/uapi/linux/userfaultfd.h
index 7d32b1e797fb..005e5e306266 100644
--- a/include/uapi/linux/userfaultfd.h
+++ b/include/uapi/linux/userfaultfd.h
@@ -12,6 +12,10 @@
#include <linux/types.h>
+/* ioctls for /dev/userfaultfd */
+#define USERFAULTFD_IOC 0xAA
+#define USERFAULTFD_IOC_NEW _IO(USERFAULTFD_IOC, 0x00)
+
/*
* If the UFFDIO_API is upgraded someday, the UFFDIO_UNREGISTER and
* UFFDIO_WAKE ioctls should be defined as _IOW and not as _IOR. In
--
2.37.0.170.g444d1eabd0-goog
On Jul 19, 2022, at 12:56 PM, Axel Rasmussen <[email protected]> wrote:
> This new mode was recently added to the userfaultfd selftest. We want to
> exercise both userfaultfd(2) as well as /dev/userfaultfd, so add both
> test cases to the script.
>
> Reviewed-by: Shuah Khan <[email protected]>
> Acked-by: Peter Xu <[email protected]>
> Signed-off-by: Axel Rasmussen <[email protected]>
> ---
> tools/testing/selftests/vm/run_vmtests.sh | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/tools/testing/selftests/vm/run_vmtests.sh b/tools/testing/selftests/vm/run_vmtests.sh
> index e70ae0f3aaf6..156f864030fc 100755
> --- a/tools/testing/selftests/vm/run_vmtests.sh
> +++ b/tools/testing/selftests/vm/run_vmtests.sh
> @@ -121,12 +121,17 @@ run_test ./gup_test -a
> run_test ./gup_test -ct -F 0x1 0 19 0x1000
>
> run_test ./userfaultfd anon 20 16
> +run_test ./userfaultfd anon:dev 20 16
> # Hugetlb tests require source and destination huge pages. Pass in half the
> # size ($half_ufd_size_MB), which is used for *each*.
> run_test ./userfaultfd hugetlb "$half_ufd_size_MB" 32
> +run_test ./userfaultfd hugetlb:dev "$half_ufd_size_MB" 32
> run_test ./userfaultfd hugetlb_shared "$half_ufd_size_MB" 32 "$mnt"/uffd-test
> rm -f "$mnt"/uffd-test
> +run_test ./userfaultfd hugetlb_shared:dev "$half_ufd_size_MB" 32 "$mnt"/uffd-test
> +rm -f "$mnt"/uffd-test
> run_test ./userfaultfd shmem 20 16
> +run_test ./userfaultfd shmem:dev 20 16
Do not do it if it would require another version of the patche-set, but
otherwise, consider using a loop as I did in [1].
[1] https://lore.kernel.org/linux-mm/[email protected]/
On Tue, Jul 19, 2022 at 12:56:25PM -0700, Axel Rasmussen wrote:
> Historically, it has been shown that intercepting kernel faults with
> userfaultfd (thereby forcing the kernel to wait for an arbitrary amount
> of time) can be exploited, or at least can make some kinds of exploits
> easier. So, in 37cd0575b8 "userfaultfd: add UFFD_USER_MODE_ONLY" we
> changed things so, in order for kernel faults to be handled by
> userfaultfd, either the process needs CAP_SYS_PTRACE, or this sysctl
> must be configured so that any unprivileged user can do it.
>
> In a typical implementation of a hypervisor with live migration (take
> QEMU/KVM as one such example), we do indeed need to be able to handle
> kernel faults. But, both options above are less than ideal:
>
> - Toggling the sysctl increases attack surface by allowing any
> unprivileged user to do it.
>
> - Granting the live migration process CAP_SYS_PTRACE gives it this
> ability, but *also* the ability to "observe and control the
> execution of another process [...], and examine and change [its]
> memory and registers" (from ptrace(2)). This isn't something we need
> or want to be able to do, so granting this permission violates the
> "principle of least privilege".
>
> This is all a long winded way to say: we want a more fine-grained way to
> grant access to userfaultfd, without granting other additional
> permissions at the same time.
>
> To achieve this, add a /dev/userfaultfd misc device. This device
> provides an alternative to the userfaultfd(2) syscall for the creation
> of new userfaultfds. The idea is, any userfaultfds created this way will
> be able to handle kernel faults, without the caller having any special
> capabilities. Access to this mechanism is instead restricted using e.g.
> standard filesystem permissions.
>
> Signed-off-by: Axel Rasmussen <[email protected]>
Thanks, this looks much better.
Acked-by: Peter Xu <[email protected]>
--
Peter Xu
On Tue, Jul 19, 2022 at 12:56:27PM -0700, Axel Rasmussen wrote:
> Explain the different ways to create a new userfaultfd, and how access
> control works for each way.
>
> Signed-off-by: Axel Rasmussen <[email protected]>
Acked-by: Peter Xu <[email protected]>
--
Peter Xu
On Tue, Jul 19, 2022 at 12:56:26PM -0700, Axel Rasmussen wrote:
> We clearly want to ensure both userfaultfd(2) and /dev/userfaultfd keep
> working into the future, so just run the test twice, using each
> interface.
>
> Instead of always testing both userfaultfd(2) and /dev/userfaultfd,
> let the user choose which to test.
>
> As with other test features, change the behavior based on a new
> command line flag. Introduce the idea of "test mods", which are
> generic (not specific to a test type) modifications to the behavior of
> the test. This is sort of borrowed from this RFC patch series [1], but
> simplified a bit.
>
> The benefit is, in "typical" configurations this test is somewhat slow
> (say, 30sec or something). Testing both clearly doubles it, so it may
> not always be desirable, as users are likely to use one or the other,
> but never both, in the "real world".
>
> [1]: https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/
>
> Signed-off-by: Axel Rasmussen <[email protected]>
Acked-by: Peter Xu <[email protected]>
--
Peter Xu
On Jul 19, 2022, at 12:56 PM, Axel Rasmussen <[email protected]> wrote:
> Historically, it has been shown that intercepting kernel faults with
> userfaultfd (thereby forcing the kernel to wait for an arbitrary amount
> of time) can be exploited, or at least can make some kinds of exploits
> easier. So, in 37cd0575b8 "userfaultfd: add UFFD_USER_MODE_ONLY" we
> changed things so, in order for kernel faults to be handled by
> userfaultfd, either the process needs CAP_SYS_PTRACE, or this sysctl
> must be configured so that any unprivileged user can do it.
>
> In a typical implementation of a hypervisor with live migration (take
> QEMU/KVM as one such example), we do indeed need to be able to handle
> kernel faults. But, both options above are less than ideal:
>
> - Toggling the sysctl increases attack surface by allowing any
> unprivileged user to do it.
>
> - Granting the live migration process CAP_SYS_PTRACE gives it this
> ability, but *also* the ability to "observe and control the
> execution of another process [...], and examine and change [its]
> memory and registers" (from ptrace(2)). This isn't something we need
> or want to be able to do, so granting this permission violates the
> "principle of least privilege".
>
> This is all a long winded way to say: we want a more fine-grained way to
> grant access to userfaultfd, without granting other additional
> permissions at the same time.
>
> To achieve this, add a /dev/userfaultfd misc device. This device
> provides an alternative to the userfaultfd(2) syscall for the creation
> of new userfaultfds. The idea is, any userfaultfds created this way will
> be able to handle kernel faults, without the caller having any special
> capabilities. Access to this mechanism is instead restricted using e.g.
> standard filesystem permissions.
Are there any other “devices" that when opened by different processes
provide such isolated interfaces in each process? I.e., devices that if you
read from them in different processes you get completely unrelated data?
(putting aside namespaces).
It all sounds so wrong to me, that I am going to try again to pushback
(sorry).
From a semantic point of view - userfaultfd is process specific. It is
therefore similar to /proc/[pid]/mem (or /proc/[pid]/pagemap and so on).
So why can’t we put it there? I saw that you argued against it in your
cover-letter, and I think that your argument is you would need
CAP_SYS_PTRACE if you want to access userfaultfd of other processes. But
this is EXACTLY the way opening /proc/[pid]/mem is performed - see
proc_mem_open().
So instead of having some strange device that behaves differently in the
context of each process, you can just have /proc/[pid]/userfaultfd and then
use mm_access() to check if you have permissions to access userfaultfd (just
like proc_mem_open() does). This would be more intuitive for users as it is
similar to other /proc/[pid]/X, and would cover both local and remote
use-cases.
On Tue, Jul 19, 2022 at 3:32 PM Nadav Amit <[email protected]> wrote:
>
> On Jul 19, 2022, at 12:56 PM, Axel Rasmussen <[email protected]> wrote:
>
> > Historically, it has been shown that intercepting kernel faults with
> > userfaultfd (thereby forcing the kernel to wait for an arbitrary amount
> > of time) can be exploited, or at least can make some kinds of exploits
> > easier. So, in 37cd0575b8 "userfaultfd: add UFFD_USER_MODE_ONLY" we
> > changed things so, in order for kernel faults to be handled by
> > userfaultfd, either the process needs CAP_SYS_PTRACE, or this sysctl
> > must be configured so that any unprivileged user can do it.
> >
> > In a typical implementation of a hypervisor with live migration (take
> > QEMU/KVM as one such example), we do indeed need to be able to handle
> > kernel faults. But, both options above are less than ideal:
> >
> > - Toggling the sysctl increases attack surface by allowing any
> > unprivileged user to do it.
> >
> > - Granting the live migration process CAP_SYS_PTRACE gives it this
> > ability, but *also* the ability to "observe and control the
> > execution of another process [...], and examine and change [its]
> > memory and registers" (from ptrace(2)). This isn't something we need
> > or want to be able to do, so granting this permission violates the
> > "principle of least privilege".
> >
> > This is all a long winded way to say: we want a more fine-grained way to
> > grant access to userfaultfd, without granting other additional
> > permissions at the same time.
> >
> > To achieve this, add a /dev/userfaultfd misc device. This device
> > provides an alternative to the userfaultfd(2) syscall for the creation
> > of new userfaultfds. The idea is, any userfaultfds created this way will
> > be able to handle kernel faults, without the caller having any special
> > capabilities. Access to this mechanism is instead restricted using e.g.
> > standard filesystem permissions.
>
> Are there any other “devices" that when opened by different processes
> provide such isolated interfaces in each process? I.e., devices that if you
> read from them in different processes you get completely unrelated data?
> (putting aside namespaces).
>
> It all sounds so wrong to me, that I am going to try again to pushback
> (sorry).
No need to be sorry. :)
>
> From a semantic point of view - userfaultfd is process specific. It is
> therefore similar to /proc/[pid]/mem (or /proc/[pid]/pagemap and so on).
>
> So why can’t we put it there? I saw that you argued against it in your
> cover-letter, and I think that your argument is you would need
> CAP_SYS_PTRACE if you want to access userfaultfd of other processes. But
> this is EXACTLY the way opening /proc/[pid]/mem is performed - see
> proc_mem_open().
>
> So instead of having some strange device that behaves differently in the
> context of each process, you can just have /proc/[pid]/userfaultfd and then
> use mm_access() to check if you have permissions to access userfaultfd (just
> like proc_mem_open() does). This would be more intuitive for users as it is
> similar to other /proc/[pid]/X, and would cover both local and remote
> use-cases.
Ah, so actually I find this argument much more compelling.
I don't find it persuasive that we should put it in /proc for the
purpose of supporting cross-process memory manipulation, because I
think the syscall works better for that, and in that case we don't
mind depending on CAP_SYS_PTRACE.
But, what you've argued here I do find persuasive. :) You are right, I
can't think of any other example of a device node in /dev that works
like this, where it is completely independent on a per-process basis.
The closest I could come up with was /dev/zero or /dev/null or
similar. You won't affect any other process by touching these, but I
don't think these are good examples.
I'll send a v5 which does this. I do worry that cross-process support
is probably complex to get right, so I might leave that out and only
allow a process to open its own device for now.
>
On Jul 19, 2022, at 3:45 PM, Axel Rasmussen <[email protected]> wrote:
> On Tue, Jul 19, 2022 at 3:32 PM Nadav Amit <[email protected]> wrote:
>> On Jul 19, 2022, at 12:56 PM, Axel Rasmussen <[email protected]> wrote:
>>
>>> Historically, it has been shown that intercepting kernel faults with
>>> userfaultfd (thereby forcing the kernel to wait for an arbitrary amount
>>> of time) can be exploited, or at least can make some kinds of exploits
>>> easier. So, in 37cd0575b8 "userfaultfd: add UFFD_USER_MODE_ONLY" we
>>> changed things so, in order for kernel faults to be handled by
>>> userfaultfd, either the process needs CAP_SYS_PTRACE, or this sysctl
>>> must be configured so that any unprivileged user can do it.
>>>
>>> In a typical implementation of a hypervisor with live migration (take
>>> QEMU/KVM as one such example), we do indeed need to be able to handle
>>> kernel faults. But, both options above are less than ideal:
>>>
>>> - Toggling the sysctl increases attack surface by allowing any
>>> unprivileged user to do it.
>>>
>>> - Granting the live migration process CAP_SYS_PTRACE gives it this
>>> ability, but *also* the ability to "observe and control the
>>> execution of another process [...], and examine and change [its]
>>> memory and registers" (from ptrace(2)). This isn't something we need
>>> or want to be able to do, so granting this permission violates the
>>> "principle of least privilege".
>>>
>>> This is all a long winded way to say: we want a more fine-grained way to
>>> grant access to userfaultfd, without granting other additional
>>> permissions at the same time.
>>>
>>> To achieve this, add a /dev/userfaultfd misc device. This device
>>> provides an alternative to the userfaultfd(2) syscall for the creation
>>> of new userfaultfds. The idea is, any userfaultfds created this way will
>>> be able to handle kernel faults, without the caller having any special
>>> capabilities. Access to this mechanism is instead restricted using e.g.
>>> standard filesystem permissions.
>>
>> Are there any other “devices" that when opened by different processes
>> provide such isolated interfaces in each process? I.e., devices that if you
>> read from them in different processes you get completely unrelated data?
>> (putting aside namespaces).
>>
>> It all sounds so wrong to me, that I am going to try again to pushback
>> (sorry).
>
> No need to be sorry. :)
>
>> From a semantic point of view - userfaultfd is process specific. It is
>> therefore similar to /proc/[pid]/mem (or /proc/[pid]/pagemap and so on).
>>
>> So why can’t we put it there? I saw that you argued against it in your
>> cover-letter, and I think that your argument is you would need
>> CAP_SYS_PTRACE if you want to access userfaultfd of other processes. But
>> this is EXACTLY the way opening /proc/[pid]/mem is performed - see
>> proc_mem_open().
>>
>> So instead of having some strange device that behaves differently in the
>> context of each process, you can just have /proc/[pid]/userfaultfd and then
>> use mm_access() to check if you have permissions to access userfaultfd (just
>> like proc_mem_open() does). This would be more intuitive for users as it is
>> similar to other /proc/[pid]/X, and would cover both local and remote
>> use-cases.
>
> Ah, so actually I find this argument much more compelling.
>
> I don't find it persuasive that we should put it in /proc for the
> purpose of supporting cross-process memory manipulation, because I
> think the syscall works better for that, and in that case we don't
> mind depending on CAP_SYS_PTRACE.
>
> But, what you've argued here I do find persuasive. :) You are right, I
> can't think of any other example of a device node in /dev that works
> like this, where it is completely independent on a per-process basis.
> The closest I could come up with was /dev/zero or /dev/null or
> similar. You won't affect any other process by touching these, but I
> don't think these are good examples.
>
> I'll send a v5 which does this. I do worry that cross-process support
> is probably complex to get right, so I might leave that out and only
> allow a process to open its own device for now.
So I didn’t want to get into it, and I am fine that you leave it out,
since such an interface would still enable to support it later.
Anyhow, I do want to clarify a bit about the “cross-process support”
userfaultfd situation. Basically, you can already get cross-process support
today, by using calling userfaultfd() on the controlled process and calling
pidfd_open() from another process. It does work and I do not remember any
issues that it introduced (in contrast, for instance, to io-uring, that
would break if you use userfaultfd+iouring+fork today).
Thanks for your reconsideration.
Regards,
Nadav
On Tue, Jul 19, 2022 at 11:55:21PM +0000, Nadav Amit wrote:
> Anyhow, I do want to clarify a bit about the “cross-process support”
> userfaultfd situation. Basically, you can already get cross-process support
> today, by using calling userfaultfd() on the controlled process and calling
> pidfd_open() from another process. It does work and I do not remember any
> issues that it introduced (in contrast, for instance, to io-uring, that
> would break if you use userfaultfd+iouring+fork today).
Do you mean to base it on pidof_getfd()?
Just want to mention that this will still need collaboration of the target
process as userfaultfd needs to be created explicitly there. From that POV
it's still more similar to general SCM_RIGHTS trick to pass over the fd but
just to pass it in a different way.
IMHO the core change about having /proc/pid/userfaultfd is skipping that
only last step to create the handle.
--
Peter Xu
On Jul 19, 2022, at 7:32 PM, Peter Xu <[email protected]> wrote:
> ⚠ External Email
>
> On Tue, Jul 19, 2022 at 11:55:21PM +0000, Nadav Amit wrote:
>> Anyhow, I do want to clarify a bit about the “cross-process support”
>> userfaultfd situation. Basically, you can already get cross-process support
>> today, by using calling userfaultfd() on the controlled process and calling
>> pidfd_open() from another process. It does work and I do not remember any
>> issues that it introduced (in contrast, for instance, to io-uring, that
>> would break if you use userfaultfd+iouring+fork today).
>
> Do you mean to base it on pidof_getfd()?
autocorrect? :)
I did refer to pidfd_getfd() as a syscall that can be used today by one
process to control the address space of another process. I did not intend to
use it for the actual implementation.
> Just want to mention that this will still need collaboration of the target
> process as userfaultfd needs to be created explicitly there. From that POV
> it's still more similar to general SCM_RIGHTS trick to pass over the fd but
> just to pass it in a different way.
There are also some tricks you can do with ptrace in order not to need the
collaboration, but they are admittedly fragile.
> IMHO the core change about having /proc/pid/userfaultfd is skipping that
> only last step to create the handle.
Yes. The point that I was trying to make is that there are no open issues
with adding support for remote process control through
/proc/pid/userfaultfd. This is in contrast, for example, for using io-uring
with userfaultfd. For instance, if you try to use io-uring TODAY with
userfaultfd (without the async support that I need to add), and you try to
monitor the fork event, things would break (the new userfaultfd file
descriptor after fork would be installed on the io-worker thread).
This is all to say that it is really simple to add support for one process
monitoring userfaultfd of another process, since I understood that Axel had
concerned that this might be utterly broken…
On Wed, Jul 20, 2022 at 10:42 AM Nadav Amit <[email protected]> wrote:
>
> On Jul 19, 2022, at 7:32 PM, Peter Xu <[email protected]> wrote:
>
> > ⚠ External Email
> >
> > On Tue, Jul 19, 2022 at 11:55:21PM +0000, Nadav Amit wrote:
> >> Anyhow, I do want to clarify a bit about the “cross-process support”
> >> userfaultfd situation. Basically, you can already get cross-process support
> >> today, by using calling userfaultfd() on the controlled process and calling
> >> pidfd_open() from another process. It does work and I do not remember any
> >> issues that it introduced (in contrast, for instance, to io-uring, that
> >> would break if you use userfaultfd+iouring+fork today).
> >
> > Do you mean to base it on pidof_getfd()?
>
> autocorrect? :)
>
> I did refer to pidfd_getfd() as a syscall that can be used today by one
> process to control the address space of another process. I did not intend to
> use it for the actual implementation.
>
> > Just want to mention that this will still need collaboration of the target
> > process as userfaultfd needs to be created explicitly there. From that POV
> > it's still more similar to general SCM_RIGHTS trick to pass over the fd but
> > just to pass it in a different way.
>
> There are also some tricks you can do with ptrace in order not to need the
> collaboration, but they are admittedly fragile.
>
> > IMHO the core change about having /proc/pid/userfaultfd is skipping that
> > only last step to create the handle.
>
> Yes. The point that I was trying to make is that there are no open issues
> with adding support for remote process control through
> /proc/pid/userfaultfd. This is in contrast, for example, for using io-uring
> with userfaultfd. For instance, if you try to use io-uring TODAY with
> userfaultfd (without the async support that I need to add), and you try to
> monitor the fork event, things would break (the new userfaultfd file
> descriptor after fork would be installed on the io-worker thread).
>
> This is all to say that it is really simple to add support for one process
> monitoring userfaultfd of another process, since I understood that Axel had
> concerned that this might be utterly broken…
Mostly I was worried it would be nontrivial to implement, and it isn't
a use case I plan to use so I was hoping to ignore it and defer it to
some future patches. ;)
But, if it "just works" I'm happy to include it in v5.
On Jul 20, 2022, at 1:10 PM, Axel Rasmussen <[email protected]> wrote:
> On Wed, Jul 20, 2022 at 10:42 AM Nadav Amit <[email protected]> wrote:
>> On Jul 19, 2022, at 7:32 PM, Peter Xu <[email protected]> wrote:
>>
>>> ⚠ External Email
>>>
>>> On Tue, Jul 19, 2022 at 11:55:21PM +0000, Nadav Amit wrote:
>>>> Anyhow, I do want to clarify a bit about the “cross-process support”
>>>> userfaultfd situation. Basically, you can already get cross-process support
>>>> today, by using calling userfaultfd() on the controlled process and calling
>>>> pidfd_open() from another process. It does work and I do not remember any
>>>> issues that it introduced (in contrast, for instance, to io-uring, that
>>>> would break if you use userfaultfd+iouring+fork today).
>>>
>>> Do you mean to base it on pidof_getfd()?
>>
>> autocorrect? :)
>>
>> I did refer to pidfd_getfd() as a syscall that can be used today by one
>> process to control the address space of another process. I did not intend to
>> use it for the actual implementation.
>>
>>> Just want to mention that this will still need collaboration of the target
>>> process as userfaultfd needs to be created explicitly there. From that POV
>>> it's still more similar to general SCM_RIGHTS trick to pass over the fd but
>>> just to pass it in a different way.
>>
>> There are also some tricks you can do with ptrace in order not to need the
>> collaboration, but they are admittedly fragile.
>>
>>> IMHO the core change about having /proc/pid/userfaultfd is skipping that
>>> only last step to create the handle.
>>
>> Yes. The point that I was trying to make is that there are no open issues
>> with adding support for remote process control through
>> /proc/pid/userfaultfd. This is in contrast, for example, for using io-uring
>> with userfaultfd. For instance, if you try to use io-uring TODAY with
>> userfaultfd (without the async support that I need to add), and you try to
>> monitor the fork event, things would break (the new userfaultfd file
>> descriptor after fork would be installed on the io-worker thread).
>>
>> This is all to say that it is really simple to add support for one process
>> monitoring userfaultfd of another process, since I understood that Axel had
>> concerned that this might be utterly broken…
>
> Mostly I was worried it would be nontrivial to implement, and it isn't
> a use case I plan to use so I was hoping to ignore it and defer it to
> some future patches. ;)
>
> But, if it "just works" I'm happy to include it in v5.
There is a problem though, since for many use-cases you do need
process_madvisev(MADV_DONTNEED) which is unsupported, and you also need - in
some use-cases - to be able to skip pinned pages. These are patches that I
still need to send.
So I leave it to you to make up your mind whether it is reasonable to add it
now without this support.
> -----Original Message-----
> From: Axel Rasmussen <[email protected]>
> Sent: Tuesday, July 19, 2022 12:56 PM
> To: Alexander Viro <[email protected]>; Andrew Morton
> <[email protected]>; Dave Hansen
> <[email protected]>; Dmitry V . Levin <[email protected]>; Gleb
> Fotengauer-Malinovskiy <[email protected]>; Hugh Dickins
> <[email protected]>; Jan Kara <[email protected]>; Jonathan Corbet
> <[email protected]>; Mel Gorman <[email protected]>; Mike
> Kravetz <[email protected]>; Mike Rapoport <[email protected]>;
> Amit, Nadav <[email protected]>; Peter Xu <[email protected]>;
> Shuah Khan <[email protected]>; Suren Baghdasaryan
> <[email protected]>; Vlastimil Babka <[email protected]>; zhangyi
> <[email protected]>
> Cc: Axel Rasmussen <[email protected]>; linux-
> [email protected]; [email protected]; linux-
> [email protected]; [email protected]; linux-
> [email protected]
> Subject: [PATCH v4 0/5] userfaultfd: add /dev/userfaultfd for fine grained
> access control
I assume that leaving the LSM mailing list off of the CC is purely
accidental. Please, please include us in the next round.
>
> This series is based on torvalds/master.
>
> The series is split up like so:
> - Patch 1 is a simple fixup which we should take in any case (even by itself).
> - Patches 2-6 add the feature, configurable selftest support, and docs.
>
> Why not ...?
> ============
>
> - Why not /proc/[pid]/userfaultfd? The proposed use case for this is for one
> process to open a userfaultfd which can intercept another process' page
> faults. This seems to me like exactly what CAP_SYS_PTRACE is for, though,
> so I
> think this use case can simply use a syscall without the powers
> CAP_SYS_PTRACE
> grants being "too much".
>
> - Why not use a syscall? Access to syscalls is generally controlled by
> capabilities. We don't have a capability which is used for userfaultfd access
> without also granting more / other permissions as well, and adding a new
> capability was rejected [1].
>
> - It's possible a LSM could be used to control access instead. I suspect
> adding a brand new one just for this would be rejected,
You won't know if you don't ask.
> but I think some
> existing ones like SELinux can be used to filter syscall access. Enabling
> SELinux for large production deployments which don't already use it is
> likely to be a huge undertaking though, and I don't think this use case by
> itself is enough to motivate that kind of architectural change.
>
> Changelog
> =========
>
> v3->v4:
> - Picked up an Acked-by on 5/5.
> - Updated cover letter to cover "why not ...".
> - Refactored userfaultfd_allowed() into userfaultfd_syscall_allowed().
> [Peter]
> - Removed obsolete comment from a previous version. [Peter]
> - Refactored userfaultfd_open() in selftest. [Peter]
> - Reworded admin-guide documentation. [Mike, Peter]
> - Squashed 2 commits adding /dev/userfaultfd to selftest and making
> selftest
> configurable. [Peter]
> - Added "syscall" test modifier (the default behavior) to selftest. [Peter]
>
> v2->v3:
> - Rebased onto linux-next/akpm-base, in order to be based on top of the
> run_vmtests.sh refactor which was merged previously.
> - Picked up some Reviewed-by's.
> - Fixed ioctl definition (_IO instead of _IOWR), and stopped using
> compat_ptr_ioctl since it is unneeded for ioctls which don't take a pointer.
> - Removed the "handle_kernel_faults" bool, simplifying the code. The result
> is
> logically equivalent, but simpler.
> - Fixed userfaultfd selftest so it returns KSFT_SKIP appropriately.
> - Reworded documentation per Shuah's feedback on v2.
> - Improved example usage for userfaultfd selftest.
>
> v1->v2:
> - Add documentation update.
> - Test *both* userfaultfd(2) and /dev/userfaultfd via the selftest.
>
> [1]: https://lore.kernel.org/lkml/686276b9-4530-2045-6bd8-
> [email protected]/T/
>
> Axel Rasmussen (5):
> selftests: vm: add hugetlb_shared userfaultfd test to run_vmtests.sh
> userfaultfd: add /dev/userfaultfd for fine grained access control
> userfaultfd: selftests: modify selftest to use /dev/userfaultfd
> userfaultfd: update documentation to describe /dev/userfaultfd
> selftests: vm: add /dev/userfaultfd test cases to run_vmtests.sh
>
> Documentation/admin-guide/mm/userfaultfd.rst | 41 +++++++++++-
> Documentation/admin-guide/sysctl/vm.rst | 3 +
> fs/userfaultfd.c | 69 ++++++++++++++++----
> include/uapi/linux/userfaultfd.h | 4 ++
> tools/testing/selftests/vm/run_vmtests.sh | 11 +++-
> tools/testing/selftests/vm/userfaultfd.c | 69 +++++++++++++++++---
> 6 files changed, 169 insertions(+), 28 deletions(-)
>
> --
> 2.37.0.170.g444d1eabd0-goog
On Jul 20, 2022, at 4:04 PM, Axel Rasmussen <[email protected]> wrote:
> ⚠ External Email
>
> On Wed, Jul 20, 2022 at 3:16 PM Schaufler, Casey
> <[email protected]> wrote:
>>> -----Original Message-----
>>> From: Axel Rasmussen <[email protected]>
>>> Sent: Tuesday, July 19, 2022 12:56 PM
>>> To: Alexander Viro <[email protected]>; Andrew Morton
>>> <[email protected]>; Dave Hansen
>>> <[email protected]>; Dmitry V . Levin <[email protected]>; Gleb
>>> Fotengauer-Malinovskiy <[email protected]>; Hugh Dickins
>>> <[email protected]>; Jan Kara <[email protected]>; Jonathan Corbet
>>> <[email protected]>; Mel Gorman <[email protected]>; Mike
>>> Kravetz <[email protected]>; Mike Rapoport <[email protected]>;
>>> Amit, Nadav <[email protected]>; Peter Xu <[email protected]>;
>>> Shuah Khan <[email protected]>; Suren Baghdasaryan
>>> <[email protected]>; Vlastimil Babka <[email protected]>; zhangyi
>>> <[email protected]>
>>> Cc: Axel Rasmussen <[email protected]>; linux-
>>> [email protected]; [email protected]; linux-
>>> [email protected]; [email protected]; linux-
>>> [email protected]
>>> Subject: [PATCH v4 0/5] userfaultfd: add /dev/userfaultfd for fine grained
>>> access control
>>
>> I assume that leaving the LSM mailing list off of the CC is purely
>> accidental. Please, please include us in the next round.
>
> Honestly it just hadn't occurred to me, but I'm more than happy to CC
> it on future revisions.
>
>>> This series is based on torvalds/master.
>>>
>>> The series is split up like so:
>>> - Patch 1 is a simple fixup which we should take in any case (even by itself).
>>> - Patches 2-6 add the feature, configurable selftest support, and docs.
>>>
>>> Why not ...?
>>> ============
>>>
>>> - Why not /proc/[pid]/userfaultfd? The proposed use case for this is for one
>>> process to open a userfaultfd which can intercept another process' page
>>> faults. This seems to me like exactly what CAP_SYS_PTRACE is for, though,
>>> so I
>>> think this use case can simply use a syscall without the powers
>>> CAP_SYS_PTRACE
>>> grants being "too much".
>>>
>>> - Why not use a syscall? Access to syscalls is generally controlled by
>>> capabilities. We don't have a capability which is used for userfaultfd access
>>> without also granting more / other permissions as well, and adding a new
>>> capability was rejected [1].
>>>
>>> - It's possible a LSM could be used to control access instead. I suspect
>>> adding a brand new one just for this would be rejected,
>>
>> You won't know if you don't ask.
>
> Fair enough - I wonder if MM folks (Andrew, Peter, Nadav especially)
> would find that approach more palatable than /proc/[pid]/userfaultfd?
> Would it make sense from our perspective to propose a userfaultfd- or
> MM-specific LSM for controlling access to certain features?
>
> I remember +Andrea saying Red Hat was also interested in some kind of
> access control mechanism like this. Would one or the other approach be
> more convenient for you?
To reiterate my position - I think that /proc/[pid]/userfaultfd is very
natural and can be easily extended to support cross-process access of
userfaultfd. The necessary access controls are simple in any case. For
cross-process access, they are similar to those that are used for other
/proc/[pid]/X such as pagemap.
I have little experience with LSM and I do not know how real deployments use
them. If they are easier to deploy and people prefer them over some
pseudo-file, I cannot argue against them.
On Wed, Jul 20, 2022 at 3:16 PM Schaufler, Casey
<[email protected]> wrote:
>
> > -----Original Message-----
> > From: Axel Rasmussen <[email protected]>
> > Sent: Tuesday, July 19, 2022 12:56 PM
> > To: Alexander Viro <[email protected]>; Andrew Morton
> > <[email protected]>; Dave Hansen
> > <[email protected]>; Dmitry V . Levin <[email protected]>; Gleb
> > Fotengauer-Malinovskiy <[email protected]>; Hugh Dickins
> > <[email protected]>; Jan Kara <[email protected]>; Jonathan Corbet
> > <[email protected]>; Mel Gorman <[email protected]>; Mike
> > Kravetz <[email protected]>; Mike Rapoport <[email protected]>;
> > Amit, Nadav <[email protected]>; Peter Xu <[email protected]>;
> > Shuah Khan <[email protected]>; Suren Baghdasaryan
> > <[email protected]>; Vlastimil Babka <[email protected]>; zhangyi
> > <[email protected]>
> > Cc: Axel Rasmussen <[email protected]>; linux-
> > [email protected]; [email protected]; linux-
> > [email protected]; [email protected]; linux-
> > [email protected]
> > Subject: [PATCH v4 0/5] userfaultfd: add /dev/userfaultfd for fine grained
> > access control
>
> I assume that leaving the LSM mailing list off of the CC is purely
> accidental. Please, please include us in the next round.
Honestly it just hadn't occurred to me, but I'm more than happy to CC
it on future revisions.
>
> >
> > This series is based on torvalds/master.
> >
> > The series is split up like so:
> > - Patch 1 is a simple fixup which we should take in any case (even by itself).
> > - Patches 2-6 add the feature, configurable selftest support, and docs.
> >
> > Why not ...?
> > ============
> >
> > - Why not /proc/[pid]/userfaultfd? The proposed use case for this is for one
> > process to open a userfaultfd which can intercept another process' page
> > faults. This seems to me like exactly what CAP_SYS_PTRACE is for, though,
> > so I
> > think this use case can simply use a syscall without the powers
> > CAP_SYS_PTRACE
> > grants being "too much".
> >
> > - Why not use a syscall? Access to syscalls is generally controlled by
> > capabilities. We don't have a capability which is used for userfaultfd access
> > without also granting more / other permissions as well, and adding a new
> > capability was rejected [1].
> >
> > - It's possible a LSM could be used to control access instead. I suspect
> > adding a brand new one just for this would be rejected,
>
> You won't know if you don't ask.
Fair enough - I wonder if MM folks (Andrew, Peter, Nadav especially)
would find that approach more palatable than /proc/[pid]/userfaultfd?
Would it make sense from our perspective to propose a userfaultfd- or
MM-specific LSM for controlling access to certain features?
I remember +Andrea saying Red Hat was also interested in some kind of
access control mechanism like this. Would one or the other approach be
more convenient for you?
>
> > but I think some
> > existing ones like SELinux can be used to filter syscall access. Enabling
> > SELinux for large production deployments which don't already use it is
> > likely to be a huge undertaking though, and I don't think this use case by
> > itself is enough to motivate that kind of architectural change.
> >
> > Changelog
> > =========
> >
> > v3->v4:
> > - Picked up an Acked-by on 5/5.
> > - Updated cover letter to cover "why not ...".
> > - Refactored userfaultfd_allowed() into userfaultfd_syscall_allowed().
> > [Peter]
> > - Removed obsolete comment from a previous version. [Peter]
> > - Refactored userfaultfd_open() in selftest. [Peter]
> > - Reworded admin-guide documentation. [Mike, Peter]
> > - Squashed 2 commits adding /dev/userfaultfd to selftest and making
> > selftest
> > configurable. [Peter]
> > - Added "syscall" test modifier (the default behavior) to selftest. [Peter]
> >
> > v2->v3:
> > - Rebased onto linux-next/akpm-base, in order to be based on top of the
> > run_vmtests.sh refactor which was merged previously.
> > - Picked up some Reviewed-by's.
> > - Fixed ioctl definition (_IO instead of _IOWR), and stopped using
> > compat_ptr_ioctl since it is unneeded for ioctls which don't take a pointer.
> > - Removed the "handle_kernel_faults" bool, simplifying the code. The result
> > is
> > logically equivalent, but simpler.
> > - Fixed userfaultfd selftest so it returns KSFT_SKIP appropriately.
> > - Reworded documentation per Shuah's feedback on v2.
> > - Improved example usage for userfaultfd selftest.
> >
> > v1->v2:
> > - Add documentation update.
> > - Test *both* userfaultfd(2) and /dev/userfaultfd via the selftest.
> >
> > [1]: https://lore.kernel.org/lkml/686276b9-4530-2045-6bd8-
> > [email protected]/T/
> >
> > Axel Rasmussen (5):
> > selftests: vm: add hugetlb_shared userfaultfd test to run_vmtests.sh
> > userfaultfd: add /dev/userfaultfd for fine grained access control
> > userfaultfd: selftests: modify selftest to use /dev/userfaultfd
> > userfaultfd: update documentation to describe /dev/userfaultfd
> > selftests: vm: add /dev/userfaultfd test cases to run_vmtests.sh
> >
> > Documentation/admin-guide/mm/userfaultfd.rst | 41 +++++++++++-
> > Documentation/admin-guide/sysctl/vm.rst | 3 +
> > fs/userfaultfd.c | 69 ++++++++++++++++----
> > include/uapi/linux/userfaultfd.h | 4 ++
> > tools/testing/selftests/vm/run_vmtests.sh | 11 +++-
> > tools/testing/selftests/vm/userfaultfd.c | 69 +++++++++++++++++---
> > 6 files changed, 169 insertions(+), 28 deletions(-)
> >
> > --
> > 2.37.0.170.g444d1eabd0-goog
>
I finished up some other work and got around to writing a v5 today,
but I ran into a problem with /proc/[pid]/userfaultfd.
Files in /proc/[pid]/* are owned by the user/group which started the
process, and they don't support being chmod'ed.
For the userfaultfd device, I think we want the following semantics:
- For UFFDs created via the device, we want to always allow handling
kernel mode faults
- For security, the device should be owned by root:root by default, so
unprivileged users don't have default access to handle kernel faults
- But, the system administrator should be able to chown/chmod it, to
grant access to handling kernel faults for this process more widely.
It could be made to work like that but I think it would involve at least:
- Special casing userfaultfd in proc_pid_make_inode
- Updating setattr/getattr for /proc/[pid] to meaningfully store and
then retrieve uid/gid different from the task's, again probably
special cased for userfautlfd since we don't want this behavior for
other files
It seems to me such a change might raise eyebrows among procfs folks.
Before I spend the time to write this up, does this seem like
something that would obviously be nack'ed?
On Wed, Jul 20, 2022 at 4:21 PM Nadav Amit <[email protected]> wrote:
>
> On Jul 20, 2022, at 4:04 PM, Axel Rasmussen <[email protected]> wrote:
>
> > ⚠ External Email
> >
> > On Wed, Jul 20, 2022 at 3:16 PM Schaufler, Casey
> > <[email protected]> wrote:
> >>> -----Original Message-----
> >>> From: Axel Rasmussen <[email protected]>
> >>> Sent: Tuesday, July 19, 2022 12:56 PM
> >>> To: Alexander Viro <[email protected]>; Andrew Morton
> >>> <[email protected]>; Dave Hansen
> >>> <[email protected]>; Dmitry V . Levin <[email protected]>; Gleb
> >>> Fotengauer-Malinovskiy <[email protected]>; Hugh Dickins
> >>> <[email protected]>; Jan Kara <[email protected]>; Jonathan Corbet
> >>> <[email protected]>; Mel Gorman <[email protected]>; Mike
> >>> Kravetz <[email protected]>; Mike Rapoport <[email protected]>;
> >>> Amit, Nadav <[email protected]>; Peter Xu <[email protected]>;
> >>> Shuah Khan <[email protected]>; Suren Baghdasaryan
> >>> <[email protected]>; Vlastimil Babka <[email protected]>; zhangyi
> >>> <[email protected]>
> >>> Cc: Axel Rasmussen <[email protected]>; linux-
> >>> [email protected]; [email protected]; linux-
> >>> [email protected]; [email protected]; linux-
> >>> [email protected]
> >>> Subject: [PATCH v4 0/5] userfaultfd: add /dev/userfaultfd for fine grained
> >>> access control
> >>
> >> I assume that leaving the LSM mailing list off of the CC is purely
> >> accidental. Please, please include us in the next round.
> >
> > Honestly it just hadn't occurred to me, but I'm more than happy to CC
> > it on future revisions.
> >
> >>> This series is based on torvalds/master.
> >>>
> >>> The series is split up like so:
> >>> - Patch 1 is a simple fixup which we should take in any case (even by itself).
> >>> - Patches 2-6 add the feature, configurable selftest support, and docs.
> >>>
> >>> Why not ...?
> >>> ============
> >>>
> >>> - Why not /proc/[pid]/userfaultfd? The proposed use case for this is for one
> >>> process to open a userfaultfd which can intercept another process' page
> >>> faults. This seems to me like exactly what CAP_SYS_PTRACE is for, though,
> >>> so I
> >>> think this use case can simply use a syscall without the powers
> >>> CAP_SYS_PTRACE
> >>> grants being "too much".
> >>>
> >>> - Why not use a syscall? Access to syscalls is generally controlled by
> >>> capabilities. We don't have a capability which is used for userfaultfd access
> >>> without also granting more / other permissions as well, and adding a new
> >>> capability was rejected [1].
> >>>
> >>> - It's possible a LSM could be used to control access instead. I suspect
> >>> adding a brand new one just for this would be rejected,
> >>
> >> You won't know if you don't ask.
> >
> > Fair enough - I wonder if MM folks (Andrew, Peter, Nadav especially)
> > would find that approach more palatable than /proc/[pid]/userfaultfd?
> > Would it make sense from our perspective to propose a userfaultfd- or
> > MM-specific LSM for controlling access to certain features?
> >
> > I remember +Andrea saying Red Hat was also interested in some kind of
> > access control mechanism like this. Would one or the other approach be
> > more convenient for you?
>
> To reiterate my position - I think that /proc/[pid]/userfaultfd is very
> natural and can be easily extended to support cross-process access of
> userfaultfd. The necessary access controls are simple in any case. For
> cross-process access, they are similar to those that are used for other
> /proc/[pid]/X such as pagemap.
>
> I have little experience with LSM and I do not know how real deployments use
> them. If they are easier to deploy and people prefer them over some
> pseudo-file, I cannot argue against them.
>
>
On Aug 1, 2022, at 10:13 AM, Axel Rasmussen <[email protected]> wrote:
> ⚠ External Email
>
> I finished up some other work and got around to writing a v5 today,
> but I ran into a problem with /proc/[pid]/userfaultfd.
>
> Files in /proc/[pid]/* are owned by the user/group which started the
> process, and they don't support being chmod'ed.
>
> For the userfaultfd device, I think we want the following semantics:
> - For UFFDs created via the device, we want to always allow handling
> kernel mode faults
> - For security, the device should be owned by root:root by default, so
> unprivileged users don't have default access to handle kernel faults
> - But, the system administrator should be able to chown/chmod it, to
> grant access to handling kernel faults for this process more widely.
>
> It could be made to work like that but I think it would involve at least:
>
> - Special casing userfaultfd in proc_pid_make_inode
> - Updating setattr/getattr for /proc/[pid] to meaningfully store and
> then retrieve uid/gid different from the task's, again probably
> special cased for userfautlfd since we don't want this behavior for
> other files
>
> It seems to me such a change might raise eyebrows among procfs folks.
> Before I spend the time to write this up, does this seem like
> something that would obviously be nack'ed?
[ Please avoid top-posting in the future ]
I have no interest in making your life harder than it should be. If you
cannot find a suitable alternative, I will not fight against it.
How about this alternative: how about following KVM usage-model?
IOW: You open /dev/userfaultfd, but this is not the file-descriptor that you
use for most operations. Instead you first issue an ioctl - similarly to
KVM_CREATE_VM - to get a file-descriptor for your specific process. You then
use this new file-descriptor to perform your operations (read/ioctl/etc).
This would make the fact that ioctls/reads from different processes refer to
different contexts (i.e., file-descriptors) much more natural.
Does it sound better?
On Mon, Aug 1, 2022 at 12:53 PM Nadav Amit <[email protected]> wrote:
>
> On Aug 1, 2022, at 10:13 AM, Axel Rasmussen <[email protected]> wrote:
>
> > ⚠ External Email
> >
> > I finished up some other work and got around to writing a v5 today,
> > but I ran into a problem with /proc/[pid]/userfaultfd.
> >
> > Files in /proc/[pid]/* are owned by the user/group which started the
> > process, and they don't support being chmod'ed.
> >
> > For the userfaultfd device, I think we want the following semantics:
> > - For UFFDs created via the device, we want to always allow handling
> > kernel mode faults
> > - For security, the device should be owned by root:root by default, so
> > unprivileged users don't have default access to handle kernel faults
> > - But, the system administrator should be able to chown/chmod it, to
> > grant access to handling kernel faults for this process more widely.
> >
> > It could be made to work like that but I think it would involve at least:
> >
> > - Special casing userfaultfd in proc_pid_make_inode
> > - Updating setattr/getattr for /proc/[pid] to meaningfully store and
> > then retrieve uid/gid different from the task's, again probably
> > special cased for userfautlfd since we don't want this behavior for
> > other files
> >
> > It seems to me such a change might raise eyebrows among procfs folks.
> > Before I spend the time to write this up, does this seem like
> > something that would obviously be nack'ed?
>
> [ Please avoid top-posting in the future ]
I will remember this. Gmail's default behavior is annoying. :/
>
> I have no interest in making your life harder than it should be. If you
> cannot find a suitable alternative, I will not fight against it.
>
> How about this alternative: how about following KVM usage-model?
>
> IOW: You open /dev/userfaultfd, but this is not the file-descriptor that you
> use for most operations. Instead you first issue an ioctl - similarly to
> KVM_CREATE_VM - to get a file-descriptor for your specific process. You then
> use this new file-descriptor to perform your operations (read/ioctl/etc).
>
> This would make the fact that ioctls/reads from different processes refer to
> different contexts (i.e., file-descriptors) much more natural.
>
> Does it sound better?
Ah, that I think is more or less what my series already proposes, if I
understand you correctly.
The usage is:
fd = open(/dev/userfaultfd) /* This FD is only useful for creating new
userfaultfds */
uffd = ioctl(fd, USERFAULTFD_IOC_NEW) /* Now you get a real uffd */
close(fd); /* No longer needed now that we have a real uffd */
/* Use uffd to register, COPY, CONTINUE, whatever */
One thing we could do now or in the future is extend
USERFAULTFD_IOC_NEW to take a pid as an argument, to support creating
uffds for remote processes.
And then we get the benefit of permissions for /dev nodes working very
naturally - they default to root, but can be configured by the
sysadmin via chown/chmod, or udev rules, or whatever.
On Aug 1, 2022, at 3:50 PM, Axel Rasmussen <[email protected]> wrote:
> ⚠ External Email
>
> On Mon, Aug 1, 2022 at 12:53 PM Nadav Amit <[email protected]> wrote:
>> On Aug 1, 2022, at 10:13 AM, Axel Rasmussen <[email protected]> wrote:
>
> Ah, that I think is more or less what my series already proposes, if I
> understand you correctly.
>
> The usage is:
>
> fd = open(/dev/userfaultfd) /* This FD is only useful for creating new
> userfaultfds */
> uffd = ioctl(fd, USERFAULTFD_IOC_NEW) /* Now you get a real uffd */
> close(fd); /* No longer needed now that we have a real uffd */
>
> /* Use uffd to register, COPY, CONTINUE, whatever */
>
> One thing we could do now or in the future is extend
> USERFAULTFD_IOC_NEW to take a pid as an argument, to support creating
> uffds for remote processes.
>
>
>
> And then we get the benefit of permissions for /dev nodes working very
> naturally - they default to root, but can be configured by the
> sysadmin via chown/chmod, or udev rules, or whatever.
Oh. Stupid me. Then yes, using the /dev/userfaultfd is in line with other
usage models, such as KVM. And reading from each file descriptor is indeed
providing different output.
On Jul 19, 2022, at 12:56 PM, Axel Rasmussen <[email protected]> wrote:
>
> +static int new_userfaultfd(bool is_syscall, int flags)
> {
> struct userfaultfd_ctx *ctx;
> int fd;
>
> - if (!sysctl_unprivileged_userfaultfd &&
> - (flags & UFFD_USER_MODE_ONLY) == 0 &&
> - !capable(CAP_SYS_PTRACE)) {
> - printk_once(KERN_WARNING "uffd: Set unprivileged_userfaultfd "
> - "sysctl knob to 1 if kernel faults must be handled "
> - "without obtaining CAP_SYS_PTRACE capability\n");
> + if (is_syscall && !userfaultfd_syscall_allowed(flags))
> return -EPERM;
> - }
>
> BUG_ON(!current->mm);
>
> @@ -2098,8 +2105,42 @@ SYSCALL_DEFINE1(userfaultfd, int, flags)
> return fd;
> }
>
> +SYSCALL_DEFINE1(userfaultfd, int, flags)
> +{
> + return new_userfaultfd(true, flags);
> +}
Not critical, but why not to put the userfaultfd_syscall_allowed() check
here? You would be able to lose the “is_syscall”.
I also had a small comment for patch 5.
But these are minor issues, so for the series:
Acked-by: Nadav Amit <[email protected]>
[ Sorry again for misunderstanding the scheme you were using is similar to
KVM and therefore reasonable. ]