This is an attempt to revive discussion after bringing it up as a reply
to the last attempt last week.
Since the problem with the previous attempt at adding getdents to
io_uring was that offset was problematic, we can just not add an offset
parameter: using different offsets is rarely useful in practice (maybe
for some cacheless userspace NFS server?) and more isn't worth the cost
of allowing arbitrary offsets: just allow rewind as a flag instead.
[happy to drop even that flag for what I care about, but that might be
useful enough on its own as io_uring rightfully has no seek API]
The new API does nothing that cannot be achieved with plain syscalls so
it shouldn't be introducing any new problem, the only downside is that
having the state in the file struct isn't very uring-ish and if a
better solution is found later that will probably require duplicating
some logic in a new flag... But that seems like it would likely be a
distant future, and this version should be usable right away.
To repeat the rationale for having getdents available from uring as it
has been a while, while I believe there can be real performance
improvements as suggested in [1], the real reason is to allow coherency
in code: applications using io_uring usually center their event loop
around the ring, and having the occasional synchronous getdents call in
there is cumbersome and hard to do properly when you consider e.g. a
slow or acting up network fs...
[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
(unfortunately the source is no longer available...)
liburing implementation:
https://github.com/martinetd/liburing/commits/getdents
(will submit properly after initial discussions here)
Previous discussion:
https://lore.kernel.org/all/[email protected]/T/#m517583f23502f32b040c819d930359313b3db00c
Signed-off-by: Dominique Martinet <[email protected]>
---
Changes in v2:
- implement NOWAIT as suggested by dchinner; I'm unable to provide
reliable benchmarks but it does indeed look positive (and makes sense)
to avoid spinning out to another thread when not required.
FWIW though, the serializing readdirs per inode argument given in v1
thread isn't valid: serialization is only done in io_prep_async_work
for regular files (REQ_F_ISREG, set from file mode through FFS_ISREG),
so dir operations aren't serialized in our case.
If I was pedantic I'd say we might want that hashing for filesystems
that don't implement interate_shared, but that info comes too late and
these filesystems should become less frequent so leaving as is.
- implement NOWAIT for kernfs and libfs to test with /sys
(the tentative patch for xfs didn't seem to work, didn't take the time
to debug)
- try to implement some EOF flag in CQE as suggested by Clay Harris,
this is less clearly cut and left as RFC.
The liburing test/getdents.t implementation has grown options to test
this flag and also try async explicitly in the latest commit:
https://github.com/martinetd/liburing/commits/getdents
- vfs_getdents split: add missing internal.h include
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Dominique Martinet (6):
fs: split off vfs_getdents function of getdents64 syscall
vfs_getdents/struct dir_context: add flags field
io_uring: add support for getdents
kernfs: implement readdir FMODE_NOWAIT
libfs: set FMODE_NOWAIT on dir open
RFC: io_uring getdents: test returning an EOF flag in CQE
fs/internal.h | 8 ++++++
fs/kernfs/dir.c | 21 ++++++++++++++-
fs/libfs.c | 10 ++++---
fs/readdir.c | 38 +++++++++++++++++++++------
include/linux/fs.h | 10 +++++++
include/uapi/linux/io_uring.h | 9 +++++++
io_uring/fs.c | 61 +++++++++++++++++++++++++++++++++++++++++++
io_uring/fs.h | 3 +++
io_uring/opdef.c | 8 ++++++
9 files changed, 156 insertions(+), 12 deletions(-)
---
base-commit: 58390c8ce1bddb6c623f62e7ed36383e7fa5c02f
change-id: 20230422-uring-getdents-2aab84d240aa
Best regards,
--
Dominique Martinet | Asmadeus
This splits off the vfs_getdents function from the getdents64 system
call.
This will allow io_uring to call the vfs_getdents function.
Co-authored-by: Stefan Roesch <[email protected]>
Signed-off-by: Dominique Martinet <[email protected]>
---
fs/internal.h | 8 ++++++++
fs/readdir.c | 34 ++++++++++++++++++++++++++--------
2 files changed, 34 insertions(+), 8 deletions(-)
diff --git a/fs/internal.h b/fs/internal.h
index bd3b2810a36b..e8ca000e6613 100644
--- a/fs/internal.h
+++ b/fs/internal.h
@@ -260,3 +260,11 @@ ssize_t __kernel_write_iter(struct file *file, struct iov_iter *from, loff_t *po
struct mnt_idmap *alloc_mnt_idmap(struct user_namespace *mnt_userns);
struct mnt_idmap *mnt_idmap_get(struct mnt_idmap *idmap);
void mnt_idmap_put(struct mnt_idmap *idmap);
+
+/*
+ * fs/readdir.c
+ */
+struct linux_dirent64;
+
+int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
+ unsigned int count);
diff --git a/fs/readdir.c b/fs/readdir.c
index 9c53edb60c03..ed0803d0011e 100644
--- a/fs/readdir.c
+++ b/fs/readdir.c
@@ -21,6 +21,7 @@
#include <linux/unistd.h>
#include <linux/compat.h>
#include <linux/uaccess.h>
+#include "internal.h"
#include <asm/unaligned.h>
@@ -351,10 +352,16 @@ static bool filldir64(struct dir_context *ctx, const char *name, int namlen,
return false;
}
-SYSCALL_DEFINE3(getdents64, unsigned int, fd,
- struct linux_dirent64 __user *, dirent, unsigned int, count)
+
+/**
+ * vfs_getdents - getdents without fdget
+ * @file : pointer to file struct of directory
+ * @dirent : pointer to user directory structure
+ * @count : size of buffer
+ */
+int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
+ unsigned int count)
{
- struct fd f;
struct getdents_callback64 buf = {
.ctx.actor = filldir64,
.count = count,
@@ -362,11 +369,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,
};
int error;
- f = fdget_pos(fd);
- if (!f.file)
- return -EBADF;
-
- error = iterate_dir(f.file, &buf.ctx);
+ error = iterate_dir(file, &buf.ctx);
if (error >= 0)
error = buf.error;
if (buf.prev_reclen) {
@@ -379,6 +382,21 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,
else
error = count - buf.count;
}
+ return error;
+}
+
+SYSCALL_DEFINE3(getdents64, unsigned int, fd,
+ struct linux_dirent64 __user *, dirent, unsigned int, count)
+{
+ struct fd f;
+ int error;
+
+ f = fdget_pos(fd);
+ if (!f.file)
+ return -EBADF;
+
+ error = vfs_getdents(f.file, dirent, count);
+
fdput_pos(f);
return error;
}
--
2.39.2
The flags will allow passing DIR_CONTEXT_F_NOWAIT to iterate()
implementations that support it (as signaled through FMODE_NWAIT
in file->f_mode)
Notes:
- considered using IOCB_NOWAIT but if we add more flags later it
would be confusing to keep track of which values are valid, use
dedicated flags
- might want to check ctx.flags & DIR_CONTEXT_F_NOWAIT is only set
when file->f_mode & FMODE_NOWAIT in iterate_dir() as e.g. WARN_ONCE?
Signed-off-by: Dominique Martinet <[email protected]>
---
fs/internal.h | 2 +-
fs/readdir.c | 6 ++++--
include/linux/fs.h | 8 ++++++++
3 files changed, 13 insertions(+), 3 deletions(-)
diff --git a/fs/internal.h b/fs/internal.h
index e8ca000e6613..0264b001d99a 100644
--- a/fs/internal.h
+++ b/fs/internal.h
@@ -267,4 +267,4 @@ void mnt_idmap_put(struct mnt_idmap *idmap);
struct linux_dirent64;
int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
- unsigned int count);
+ unsigned int count, unsigned long flags);
diff --git a/fs/readdir.c b/fs/readdir.c
index ed0803d0011e..1311b89d75e1 100644
--- a/fs/readdir.c
+++ b/fs/readdir.c
@@ -358,12 +358,14 @@ static bool filldir64(struct dir_context *ctx, const char *name, int namlen,
* @file : pointer to file struct of directory
* @dirent : pointer to user directory structure
* @count : size of buffer
+ * @flags : additional dir_context flags
*/
int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
- unsigned int count)
+ unsigned int count, unsigned long flags)
{
struct getdents_callback64 buf = {
.ctx.actor = filldir64,
+ .ctx.flags = flags,
.count = count,
.current_dir = dirent
};
@@ -395,7 +397,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,
if (!f.file)
return -EBADF;
- error = vfs_getdents(f.file, dirent, count);
+ error = vfs_getdents(f.file, dirent, count, 0);
fdput_pos(f);
return error;
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 21a981680856..f7de2b5ca38e 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1716,8 +1716,16 @@ typedef bool (*filldir_t)(struct dir_context *, const char *, int, loff_t, u64,
struct dir_context {
filldir_t actor;
loff_t pos;
+ unsigned long flags;
};
+/*
+ * flags for dir_context flags
+ * DIR_CONTEXT_F_NOWAIT: Request non-blocking iterate
+ * (requires file->f_mode & FMODE_NOWAIT)
+ */
+#define DIR_CONTEXT_F_NOWAIT 0x1
+
/*
* These flags let !MMU mmap() govern direct device mapping vs immediate
* copying more easily for MAP_PRIVATE, especially for ROM filesystems.
--
2.39.2
the readdir can technically wait a bit on a spinlock, but that should
never wait for long enough to return EAGAIN -- just set the capability
flag on directories f_mode
Signed-off-by: Dominique Martinet <[email protected]>
---
fs/libfs.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/libfs.c b/fs/libfs.c
index 89cf614a3271..a3c7e42d90a7 100644
--- a/fs/libfs.c
+++ b/fs/libfs.c
@@ -81,6 +81,7 @@ EXPORT_SYMBOL(simple_lookup);
int dcache_dir_open(struct inode *inode, struct file *file)
{
file->private_data = d_alloc_cursor(file->f_path.dentry);
+ file->f_mode |= FMODE_NOWAIT;
return file->private_data ? 0 : -ENOMEM;
}
--
2.39.2
Since down_read can block, use the _trylock variant if NOWAIT variant
has been requested.
(can probably do a little bit better style-wise)
Signed-off-by: Dominique Martinet <[email protected]>
---
fs/kernfs/dir.c | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c
index 45b6919903e6..5a5b3e7881bf 100644
--- a/fs/kernfs/dir.c
+++ b/fs/kernfs/dir.c
@@ -1824,7 +1824,12 @@ static int kernfs_fop_readdir(struct file *file, struct dir_context *ctx)
return 0;
root = kernfs_root(parent);
- down_read(&root->kernfs_rwsem);
+ if (ctx->flags & DIR_CONTEXT_F_NOWAIT) {
+ if (!down_read_trylock(&root->kernfs_rwsem))
+ return -EAGAIN;
+ } else {
+ down_read(&root->kernfs_rwsem);
+ }
if (kernfs_ns_enabled(parent))
ns = kernfs_info(dentry->d_sb)->ns;
@@ -1845,6 +1850,12 @@ static int kernfs_fop_readdir(struct file *file, struct dir_context *ctx)
if (!dir_emit(ctx, name, len, ino, type))
return 0;
down_read(&root->kernfs_rwsem);
+ if (ctx->flags & DIR_CONTEXT_F_NOWAIT) {
+ if (!down_read_trylock(&root->kernfs_rwsem))
+ return 0;
+ } else {
+ down_read(&root->kernfs_rwsem);
+ }
}
up_read(&root->kernfs_rwsem);
file->private_data = NULL;
@@ -1852,7 +1863,14 @@ static int kernfs_fop_readdir(struct file *file, struct dir_context *ctx)
return 0;
}
+static int kernfs_fop_dir_open(struct inode *inode, struct file *file)
+{
+ file->f_mode |= FMODE_NOWAIT;
+ return 0;
+}
+
const struct file_operations kernfs_dir_fops = {
+ .open = kernfs_fop_dir_open,
.read = generic_read_dir,
.iterate_shared = kernfs_fop_readdir,
.release = kernfs_dir_fop_release,
--
2.39.2
This turns out to be very slightly faster than an extra call to
getdents, but in practice it doesn't seem to be such an improvement as
the trailing getdents will return almost immediately be absorbed by the
scheduling noise in a find-like context (my ""server"" is too noisy to
get proper benchmarks out, but results look slightly better with this in
async mode, and almost identical in the NOWAIT path)
If the user is waiting the end of a single directory though it might be
worth it, so including the patch for comments.
(in particular I'm not really happy that the flag has become in-out for
vfs_getdents, especially when the getdents64 syscall does not use it,
but I don't see much other way around it)
If this approach is acceptable/wanted then this patch will be split down
further (at least dir_context/vfs_getdents, kernfs, libfs, uring in four
separate commits)
Signed-off-by: Dominique Martinet <[email protected]>
---
fs/internal.h | 2 +-
fs/kernfs/dir.c | 1 +
fs/libfs.c | 9 ++++++---
fs/readdir.c | 10 ++++++----
include/linux/fs.h | 2 ++
include/uapi/linux/io_uring.h | 2 ++
io_uring/fs.c | 8 ++++++--
7 files changed, 24 insertions(+), 10 deletions(-)
diff --git a/fs/internal.h b/fs/internal.h
index 0264b001d99a..0b1552c7a870 100644
--- a/fs/internal.h
+++ b/fs/internal.h
@@ -267,4 +267,4 @@ void mnt_idmap_put(struct mnt_idmap *idmap);
struct linux_dirent64;
int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
- unsigned int count, unsigned long flags);
+ unsigned int count, unsigned long *flags);
diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c
index 5a5b3e7881bf..53a6b4804c34 100644
--- a/fs/kernfs/dir.c
+++ b/fs/kernfs/dir.c
@@ -1860,6 +1860,7 @@ static int kernfs_fop_readdir(struct file *file, struct dir_context *ctx)
up_read(&root->kernfs_rwsem);
file->private_data = NULL;
ctx->pos = INT_MAX;
+ ctx->flags |= DIR_CONTEXT_F_EOD;
return 0;
}
diff --git a/fs/libfs.c b/fs/libfs.c
index a3c7e42d90a7..b2a95dadffbd 100644
--- a/fs/libfs.c
+++ b/fs/libfs.c
@@ -208,10 +208,12 @@ int dcache_readdir(struct file *file, struct dir_context *ctx)
p = &next->d_child;
}
spin_lock(&dentry->d_lock);
- if (next)
+ if (next) {
list_move_tail(&cursor->d_child, &next->d_child);
- else
+ } else {
list_del_init(&cursor->d_child);
+ ctx->flags |= DIR_CONTEXT_F_EOD;
+ }
spin_unlock(&dentry->d_lock);
dput(next);
@@ -1347,7 +1349,8 @@ static loff_t empty_dir_llseek(struct file *file, loff_t offset, int whence)
static int empty_dir_readdir(struct file *file, struct dir_context *ctx)
{
- dir_emit_dots(file, ctx);
+ if (dir_emit_dots(file, ctx))
+ ctx->flags |= DIR_CONTEXT_F_EOD;
return 0;
}
diff --git a/fs/readdir.c b/fs/readdir.c
index 1311b89d75e1..be75a2154b4f 100644
--- a/fs/readdir.c
+++ b/fs/readdir.c
@@ -358,14 +358,14 @@ static bool filldir64(struct dir_context *ctx, const char *name, int namlen,
* @file : pointer to file struct of directory
* @dirent : pointer to user directory structure
* @count : size of buffer
- * @flags : additional dir_context flags
+ * @flags : pointer to additional dir_context flags
*/
int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
- unsigned int count, unsigned long flags)
+ unsigned int count, unsigned long *flags)
{
struct getdents_callback64 buf = {
.ctx.actor = filldir64,
- .ctx.flags = flags,
+ .ctx.flags = flags ? *flags : 0,
.count = count,
.current_dir = dirent
};
@@ -384,6 +384,8 @@ int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
else
error = count - buf.count;
}
+ if (flags)
+ *flags = buf.ctx.flags;
return error;
}
@@ -397,7 +399,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,
if (!f.file)
return -EBADF;
- error = vfs_getdents(f.file, dirent, count, 0);
+ error = vfs_getdents(f.file, dirent, count, NULL);
fdput_pos(f);
return error;
diff --git a/include/linux/fs.h b/include/linux/fs.h
index f7de2b5ca38e..d1e31bccfb4f 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1723,8 +1723,10 @@ struct dir_context {
* flags for dir_context flags
* DIR_CONTEXT_F_NOWAIT: Request non-blocking iterate
* (requires file->f_mode & FMODE_NOWAIT)
+ * DIR_CONTEXT_F_EOD: Signal directory has been fully iterated, set by the fs
*/
#define DIR_CONTEXT_F_NOWAIT 0x1
+#define DIR_CONTEXT_F_EOD 0x2
/*
* These flags let !MMU mmap() govern direct device mapping vs immediate
diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
index 35d0de18d893..35877132027e 100644
--- a/include/uapi/linux/io_uring.h
+++ b/include/uapi/linux/io_uring.h
@@ -381,11 +381,13 @@ struct io_uring_cqe {
* IORING_CQE_F_SOCK_NONEMPTY If set, more data to read after socket recv
* IORING_CQE_F_NOTIF Set for notification CQEs. Can be used to distinct
* them from sends.
+ * IORING_CQE_F_EOF If set, file or directory has reached end of file.
*/
#define IORING_CQE_F_BUFFER (1U << 0)
#define IORING_CQE_F_MORE (1U << 1)
#define IORING_CQE_F_SOCK_NONEMPTY (1U << 2)
#define IORING_CQE_F_NOTIF (1U << 3)
+#define IORING_CQE_F_EOF (1U << 4)
enum {
IORING_CQE_BUFFER_SHIFT = 16,
diff --git a/io_uring/fs.c b/io_uring/fs.c
index b15ec81c1ed2..f6222b0148ef 100644
--- a/io_uring/fs.c
+++ b/io_uring/fs.c
@@ -322,6 +322,7 @@ int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
{
struct io_getdents *gd = io_kiocb_to_cmd(req, struct io_getdents);
unsigned long getdents_flags = 0;
+ u32 cqe_flags = 0;
int ret;
if (issue_flags & IO_URING_F_NONBLOCK) {
@@ -338,13 +339,16 @@ int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
goto out;
}
- ret = vfs_getdents(req->file, gd->dirent, gd->count, getdents_flags);
+ ret = vfs_getdents(req->file, gd->dirent, gd->count, &getdents_flags);
out:
if (ret == -EAGAIN &&
(issue_flags & IO_URING_F_NONBLOCK))
return -EAGAIN;
- io_req_set_res(req, ret, 0);
+ if (getdents_flags & DIR_CONTEXT_F_EOD)
+ cqe_flags |= IORING_CQE_F_EOF;
+
+ io_req_set_res(req, ret, cqe_flags);
return 0;
}
--
2.39.2
This add support for getdents64 to io_uring, acting exactly like the
syscall: the directory is iterated from it's current's position as
stored in the file struct, and the file's position is updated exactly as
if getdents64 had been called.
Additionally, since io_uring has no way of issuing a seek, a flag
IORING_GETDENTS_REWIND has been added that will seek to the start of the
directory like rewinddir(3) for users that might require such a thing.
This will act exactly as if seek then getdents64 have been called
sequentially with no atomicity guarantee:
if this wasn't clear it is the responsibility of the caller to not use
getdents multiple time on a single file in parallel if they want useful
results, as is currently the case when using the syscall from multiple
threads.
For filesystems that support NOWAIT in iterate_shared(), try to use it
first; if a user already knows the filesystem they use do not support
nowait they can force async through IOSQE_ASYNC in the sqe flags,
avoiding the need to bounce back through a useless EAGAIN return.
(Note we already do that in prep if rewind is requested)
Signed-off-by: Dominique Martinet <[email protected]>
---
include/uapi/linux/io_uring.h | 7 ++++++
io_uring/fs.c | 57 +++++++++++++++++++++++++++++++++++++++++++
io_uring/fs.h | 3 +++
io_uring/opdef.c | 8 ++++++
4 files changed, 75 insertions(+)
diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
index 0716cb17e436..35d0de18d893 100644
--- a/include/uapi/linux/io_uring.h
+++ b/include/uapi/linux/io_uring.h
@@ -65,6 +65,7 @@ struct io_uring_sqe {
__u32 xattr_flags;
__u32 msg_ring_flags;
__u32 uring_cmd_flags;
+ __u32 getdents_flags;
};
__u64 user_data; /* data to be passed back at completion time */
/* pack this to avoid bogus arm OABI complaints */
@@ -223,6 +224,7 @@ enum io_uring_op {
IORING_OP_URING_CMD,
IORING_OP_SEND_ZC,
IORING_OP_SENDMSG_ZC,
+ IORING_OP_GETDENTS,
/* this goes last, obviously */
IORING_OP_LAST,
@@ -259,6 +261,11 @@ enum io_uring_op {
*/
#define SPLICE_F_FD_IN_FIXED (1U << 31) /* the last bit of __u32 */
+/*
+ * sqe->getdents_flags
+ */
+#define IORING_GETDENTS_REWIND (1U << 0)
+
/*
* POLL_ADD flags. Note that since sqe->poll_events is the flag space, the
* command flags for POLL_ADD are stored in sqe->len.
diff --git a/io_uring/fs.c b/io_uring/fs.c
index f6a69a549fd4..b15ec81c1ed2 100644
--- a/io_uring/fs.c
+++ b/io_uring/fs.c
@@ -47,6 +47,13 @@ struct io_link {
int flags;
};
+struct io_getdents {
+ struct file *file;
+ struct linux_dirent64 __user *dirent;
+ unsigned int count;
+ int flags;
+};
+
int io_renameat_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
{
struct io_rename *ren = io_kiocb_to_cmd(req, struct io_rename);
@@ -291,3 +298,53 @@ void io_link_cleanup(struct io_kiocb *req)
putname(sl->oldpath);
putname(sl->newpath);
}
+
+int io_getdents_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
+{
+ struct io_getdents *gd = io_kiocb_to_cmd(req, struct io_getdents);
+
+ if (READ_ONCE(sqe->off) != 0)
+ return -EINVAL;
+
+ gd->dirent = u64_to_user_ptr(READ_ONCE(sqe->addr));
+ gd->count = READ_ONCE(sqe->len);
+ gd->flags = READ_ONCE(sqe->getdents_flags);
+ if (gd->flags & ~IORING_GETDENTS_REWIND)
+ return -EINVAL;
+ /* rewind cannot be nowait */
+ if (gd->flags & IORING_GETDENTS_REWIND)
+ req->flags |= REQ_F_FORCE_ASYNC;
+
+ return 0;
+}
+
+int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
+{
+ struct io_getdents *gd = io_kiocb_to_cmd(req, struct io_getdents);
+ unsigned long getdents_flags = 0;
+ int ret;
+
+ if (issue_flags & IO_URING_F_NONBLOCK) {
+ if (!(req->file->f_mode & FMODE_NOWAIT))
+ return -EAGAIN;
+
+ getdents_flags = DIR_CONTEXT_F_NOWAIT;
+ }
+ if ((gd->flags & IORING_GETDENTS_REWIND)) {
+ WARN_ON_ONCE(issue_flags & IO_URING_F_NONBLOCK);
+
+ ret = vfs_llseek(req->file, 0, SEEK_SET);
+ if (ret < 0)
+ goto out;
+ }
+
+ ret = vfs_getdents(req->file, gd->dirent, gd->count, getdents_flags);
+out:
+ if (ret == -EAGAIN &&
+ (issue_flags & IO_URING_F_NONBLOCK))
+ return -EAGAIN;
+
+ io_req_set_res(req, ret, 0);
+ return 0;
+}
+
diff --git a/io_uring/fs.h b/io_uring/fs.h
index 0bb5efe3d6bb..f83a6f3a678d 100644
--- a/io_uring/fs.h
+++ b/io_uring/fs.h
@@ -18,3 +18,6 @@ int io_symlinkat(struct io_kiocb *req, unsigned int issue_flags);
int io_linkat_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe);
int io_linkat(struct io_kiocb *req, unsigned int issue_flags);
void io_link_cleanup(struct io_kiocb *req);
+
+int io_getdents_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe);
+int io_getdents(struct io_kiocb *req, unsigned int issue_flags);
diff --git a/io_uring/opdef.c b/io_uring/opdef.c
index cca7c5b55208..8f770c582ab3 100644
--- a/io_uring/opdef.c
+++ b/io_uring/opdef.c
@@ -428,6 +428,11 @@ const struct io_issue_def io_issue_defs[] = {
.prep = io_eopnotsupp_prep,
#endif
},
+ [IORING_OP_GETDENTS] = {
+ .needs_file = 1,
+ .prep = io_getdents_prep,
+ .issue = io_getdents,
+ },
};
@@ -648,6 +653,9 @@ const struct io_cold_def io_cold_defs[] = {
.fail = io_sendrecv_fail,
#endif
},
+ [IORING_OP_GETDENTS] = {
+ .name = "GETDENTS",
+ },
};
const char *io_uring_get_opcode(u8 opcode)
--
2.39.2
Hi Dominique,
kernel test robot noticed the following build warnings:
url: https://github.com/intel-lab-lkp/linux/commits/Dominique-Martinet/fs-split-off-vfs_getdents-function-of-getdents64-syscall/20230510-185542
base: 58390c8ce1bddb6c623f62e7ed36383e7fa5c02f
patch link: https://lore.kernel.org/r/20230422-uring-getdents-v2-4-2db1e37dc55e%40codewreck.org
patch subject: [PATCH v2 4/6] kernfs: implement readdir FMODE_NOWAIT
config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20230511/[email protected]/config)
compiler: gcc-11 (Debian 11.3.0-12) 11.3.0
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Reported-by: Dan Carpenter <[email protected]>
| Link: https://lore.kernel.org/r/[email protected]/
smatch warnings:
fs/kernfs/dir.c:1863 kernfs_fop_readdir() warn: inconsistent returns '&root->kernfs_rwsem'.
vim +1863 fs/kernfs/dir.c
c637b8acbe079e Tejun Heo 2013-12-11 1815 static int kernfs_fop_readdir(struct file *file, struct dir_context *ctx)
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1816 {
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1817 struct dentry *dentry = file->f_path.dentry;
319ba91d352a74 Shaohua Li 2017-07-12 1818 struct kernfs_node *parent = kernfs_dentry_node(dentry);
324a56e16e44ba Tejun Heo 2013-12-11 1819 struct kernfs_node *pos = file->private_data;
393c3714081a53 Minchan Kim 2021-11-18 1820 struct kernfs_root *root;
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1821 const void *ns = NULL;
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1822
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1823 if (!dir_emit_dots(file, ctx))
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1824 return 0;
393c3714081a53 Minchan Kim 2021-11-18 1825
393c3714081a53 Minchan Kim 2021-11-18 1826 root = kernfs_root(parent);
a551138c4b3b9f Dominique Martinet 2023-05-10 1827 if (ctx->flags & DIR_CONTEXT_F_NOWAIT) {
a551138c4b3b9f Dominique Martinet 2023-05-10 1828 if (!down_read_trylock(&root->kernfs_rwsem))
a551138c4b3b9f Dominique Martinet 2023-05-10 1829 return -EAGAIN;
a551138c4b3b9f Dominique Martinet 2023-05-10 1830 } else {
393c3714081a53 Minchan Kim 2021-11-18 1831 down_read(&root->kernfs_rwsem);
a551138c4b3b9f Dominique Martinet 2023-05-10 1832 }
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1833
324a56e16e44ba Tejun Heo 2013-12-11 1834 if (kernfs_ns_enabled(parent))
c525aaddc366df Tejun Heo 2013-12-11 1835 ns = kernfs_info(dentry->d_sb)->ns;
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1836
c637b8acbe079e Tejun Heo 2013-12-11 1837 for (pos = kernfs_dir_pos(ns, parent, ctx->pos, pos);
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1838 pos;
c637b8acbe079e Tejun Heo 2013-12-11 1839 pos = kernfs_dir_next_pos(ns, parent, ctx->pos, pos)) {
adc5e8b58f4886 Tejun Heo 2013-12-11 1840 const char *name = pos->name;
364595a6851bf6 Jeff Layton 2023-03-30 1841 unsigned int type = fs_umode_to_dtype(pos->mode);
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1842 int len = strlen(name);
67c0496e87d193 Tejun Heo 2019-11-04 1843 ino_t ino = kernfs_ino(pos);
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1844
adc5e8b58f4886 Tejun Heo 2013-12-11 1845 ctx->pos = pos->hash;
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1846 file->private_data = pos;
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1847 kernfs_get(pos);
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1848
393c3714081a53 Minchan Kim 2021-11-18 1849 up_read(&root->kernfs_rwsem);
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1850 if (!dir_emit(ctx, name, len, ino, type))
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1851 return 0;
393c3714081a53 Minchan Kim 2021-11-18 1852 down_read(&root->kernfs_rwsem);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Needs to be deleted.
a551138c4b3b9f Dominique Martinet 2023-05-10 1853 if (ctx->flags & DIR_CONTEXT_F_NOWAIT) {
a551138c4b3b9f Dominique Martinet 2023-05-10 1854 if (!down_read_trylock(&root->kernfs_rwsem))
a551138c4b3b9f Dominique Martinet 2023-05-10 1855 return 0;
It's a bit strange the this doesn't return -EAGAIN;
a551138c4b3b9f Dominique Martinet 2023-05-10 1856 } else {
a551138c4b3b9f Dominique Martinet 2023-05-10 1857 down_read(&root->kernfs_rwsem);
a551138c4b3b9f Dominique Martinet 2023-05-10 1858 }
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1859 }
393c3714081a53 Minchan Kim 2021-11-18 1860 up_read(&root->kernfs_rwsem);
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1861 file->private_data = NULL;
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1862 ctx->pos = INT_MAX;
fd7b9f7b9776b1 Tejun Heo 2013-11-28 @1863 return 0;
fd7b9f7b9776b1 Tejun Heo 2013-11-28 1864 }
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
Dan Carpenter wrote on Thu, May 11, 2023 at 01:55:57PM +0300:
> fd7b9f7b9776b1 Tejun Heo 2013-11-28 1850 if (!dir_emit(ctx, name, len, ino, type))
> fd7b9f7b9776b1 Tejun Heo 2013-11-28 1851 return 0;
> 393c3714081a53 Minchan Kim 2021-11-18 1852 down_read(&root->kernfs_rwsem);
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> Needs to be deleted.
Uh, yes, sorry; I'm not sure how I let that slip, I guess I didn't hit
any dead lock because nothing ever tried to take a write lock after
getdents...
Thanks!
I expect there'll be other comments (this might not make it at all), so
I'll keep the v3 of this patch with this fix locally and resend after
other comments.
> a551138c4b3b9f Dominique Martinet 2023-05-10 1853 if (ctx->flags & DIR_CONTEXT_F_NOWAIT) {
> a551138c4b3b9f Dominique Martinet 2023-05-10 1854 if (!down_read_trylock(&root->kernfs_rwsem))
> a551138c4b3b9f Dominique Martinet 2023-05-10 1855 return 0;
>
> It's a bit strange the this doesn't return -EAGAIN;
That is on purpose: the getdents did work (dir_emit returned success at
least once), so the caller can process whatever was filled in the buffer
before calling iterate_shared() again.
If we were to return -EAGAIN here, we'd actually be throwing out the
entries we just filled in, and that's not what we want.
--
Dominique Martinet | Asmadeus
Hello,
kernel test robot noticed "INFO:task_blocked_for_more_than#seconds" on:
commit: a551138c4b3b9fd7e74a55d6074a013bab6246db ("[PATCH v2 4/6] kernfs: implement readdir FMODE_NOWAIT")
url: https://github.com/intel-lab-lkp/linux/commits/Dominique-Martinet/fs-split-off-vfs_getdents-function-of-getdents64-syscall/20230510-185542
patch link: https://lore.kernel.org/all/[email protected]/
patch subject: [PATCH v2 4/6] kernfs: implement readdir FMODE_NOWAIT
in testcase: boot
compiler: gcc-11
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G
(please refer to attached dmesg/kmsg for entire log/backtrace)
If you fix the issue, kindly add following tag
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-lkp/[email protected]
[ 989.795384][ T32] INFO: task systemd:1 blocked for more than 491 seconds.
[ 989.797105][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 989.798652][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.805157][ T32] task:systemd state:D stack:0 pid:1 ppid:0 flags:0x00000000
[ 989.807376][ T32] Call Trace:
[ 989.807873][ T32] <TASK>
[ 989.808312][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 989.809423][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 989.810300][ T32] ? is_bpf_text_address (arch/x86/include/asm/preempt.h:85 include/linux/rcupdate.h:99 include/linux/rcupdate.h:805 kernel/bpf/core.c:721)
[ 989.812634][ T32] ? __kernel_text_address (kernel/extable.c:79)
[ 989.813489][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:26)
[ 989.814106][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414)
[ 989.814806][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 989.815384][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 989.816060][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073)
[ 989.816862][ T32] ? filter_irq_stacks (kernel/stacktrace.c:114)
[ 989.817552][ T32] ? down_write_killable (kernel/locking/rwsem.c:997)
[ 989.818224][ T32] ? kasan_save_stack (mm/kasan/common.c:46)
[ 989.818982][ T32] ? kasan_set_track (mm/kasan/common.c:52)
[ 989.820158][ T32] ? do_filp_open (fs/namei.c:601 fs/namei.c:612 fs/namei.c:3817)
[ 989.821366][ T32] ? do_sys_openat2 (fs/open.c:1356)
[ 989.822619][ T32] ? __x64_sys_openat (fs/open.c:1383)
[ 989.823954][ T32] ? do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 989.825170][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 989.826651][ T32] down_read (kernel/locking/rwsem.c:1518)
[ 989.827317][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518)
[ 989.828033][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86)
[ 989.828720][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265)
[ 989.829326][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135)
[ 989.830424][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651)
[ 989.831513][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294)
[ 989.832733][ T32] walk_component (fs/namei.c:1994)
[ 989.833602][ T32] link_path_walk+0x533/0xa00
[ 989.834386][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243)
[ 989.835107][ T32] ? __mutex_init (arch/x86/include/asm/atomic.h:41 include/linux/atomic/atomic-instrumented.h:42 include/linux/osq_lock.h:30 kernel/locking/mutex.c:52)
[ 989.835787][ T32] ? __alloc_file (fs/file_table.c:154)
[ 989.836454][ T32] path_openat (fs/namei.c:2250 (discriminator 2) fs/namei.c:3787 (discriminator 2))
[ 989.837123][ T32] ? vfs_tmpfile_open (fs/namei.c:3773)
[ 989.837759][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 989.838514][ T32] do_filp_open (fs/namei.c:3818)
[ 989.839286][ T32] ? __update_load_avg_se (kernel/sched/pelt.c:118 kernel/sched/pelt.c:226 kernel/sched/pelt.c:308)
[ 989.840569][ T32] ? may_open_dev (fs/namei.c:3812)
[ 989.841576][ T32] ? update_load_avg (kernel/sched/fair.c:3920 kernel/sched/fair.c:4255)
[ 989.842373][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 989.843492][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153)
[ 989.844732][ T32] ? alloc_fd (arch/x86/include/asm/bitops.h:68 include/asm-generic/bitops/instrumented-non-atomic.h:29 fs/file.c:251 fs/file.c:540)
[ 989.845759][ T32] do_sys_openat2 (fs/open.c:1356)
[ 989.846384][ T32] ? build_open_flags (fs/open.c:1342)
[ 989.847049][ T32] __x64_sys_openat (fs/open.c:1383)
[ 989.847689][ T32] ? __ia32_compat_sys_open (fs/open.c:1383)
[ 989.848385][ T32] ? schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 989.848957][ T32] ? switch_fpu_return (arch/x86/include/asm/bitops.h:75 include/asm-generic/bitops/instrumented-atomic.h:42 include/linux/thread_info.h:94 arch/x86/kernel/fpu/context.h:80 arch/x86/kernel/fpu/core.c:752)
[ 989.849627][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 989.850219][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 989.850964][ T32] RIP: 0033:0x7f5c316f8be7
[ 989.851562][ T32] RSP: 002b:00007ffc354a1f10 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
[ 989.852586][ T32] RAX: ffffffffffffffda RBX: 0000562309e04d60 RCX: 00007f5c316f8be7
[ 989.853582][ T32] RDX: 0000000000080000 RSI: 0000562309eb1320 RDI: 00000000ffffff9c
[ 989.854556][ T32] RBP: 0000562309eb1320 R08: 0000000000000008 R09: 0000000000000001
[ 989.855573][ T32] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000080000
[ 989.856562][ T32] R13: 0000562309e04d60 R14: 0000000000000001 R15: 00007ffc354a2320
[ 989.857561][ T32] </TASK>
[ 989.858016][ T32] INFO: task modprobe:90 blocked for more than 491 seconds.
[ 989.859063][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 989.859862][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.861044][ T32] task:modprobe state:D stack:0 pid:90 ppid:1 flags:0x00004002
[ 989.862166][ T32] Call Trace:
[ 989.862649][ T32] <TASK>
[ 989.863098][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 989.863704][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 989.864384][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 989.865125][ T32] ? idr_get_free (arch/x86/include/asm/bitops.h:228 arch/x86/include/asm/bitops.h:240 include/asm-generic/bitops/instrumented-non-atomic.h:142 lib/radix-tree.c:113 lib/radix-tree.c:1518)
[ 989.865743][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414)
[ 989.866406][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 989.866974][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 989.867669][ T32] rwsem_down_write_slowpath (arch/x86/include/asm/current.h:41 kernel/locking/rwsem.c:1180)
[ 989.868395][ T32] ? down_timeout (kernel/locking/rwsem.c:1108)
[ 989.869015][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 989.869640][ T32] ? __kernfs_new_node (fs/kernfs/dir.c:651)
[ 989.870566][ T32] ? stack_trace_save (kernel/stacktrace.c:123)
[ 989.871721][ T32] down_write (kernel/locking/rwsem.c:1306 kernel/locking/rwsem.c:1315 kernel/locking/rwsem.c:1574)
[ 989.872459][ T32] ? rwsem_down_write_slowpath (kernel/locking/rwsem.c:1571)
[ 989.873626][ T32] ? kasan_save_stack (mm/kasan/common.c:47)
[ 989.874858][ T32] ? kasan_save_stack (mm/kasan/common.c:46)
[ 989.876064][ T32] kernfs_add_one (include/linux/kernfs.h:391 fs/kernfs/dir.c:754)
[ 989.876931][ T32] ? pcpu_chunk_refresh_hint (mm/percpu-internal.h:114 (discriminator 3) mm/percpu.c:762 (discriminator 3))
[ 989.877791][ T32] kernfs_create_dir_ns (fs/kernfs/dir.c:1044)
[ 989.878954][ T32] sysfs_create_dir_ns (fs/sysfs/dir.c:61)
[ 989.879719][ T32] ? sysfs_create_mount_point (fs/sysfs/dir.c:41)
[ 989.880421][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 989.881343][ T32] ? __kmalloc_node_track_caller (include/linux/kasan.h:196 mm/slab_common.c:966 mm/slab_common.c:986)
[ 989.882220][ T32] kobject_add_internal (lib/kobject.c:65 lib/kobject.c:233)
[ 989.882902][ T32] kobject_init_and_add (lib/kobject.c:368 lib/kobject.c:451)
[ 989.883569][ T32] ? kobject_create_and_add (lib/kobject.c:444)
[ 989.884262][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 989.885122][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153)
[ 989.885929][ T32] ? srcu_module_notify (kernel/rcu/srcutree.c:1921 kernel/rcu/srcutree.c:1954)
[ 989.889953][ T32] ? tracepoint_module_notify (kernel/tracepoint.c:664 kernel/tracepoint.c:709 kernel/tracepoint.c:701)
[ 989.890685][ T32] mod_sysfs_setup (kernel/module/sysfs.c:361 kernel/module/sysfs.c:377)
[ 989.891350][ T32] ? module_add_modinfo_attrs (kernel/module/sysfs.c:374)
[ 989.892053][ T32] ? atomic_notifier_call_chain (kernel/notifier.c:343)
[ 989.892766][ T32] ? klp_module_coming (kernel/livepatch/core.c:1296)
[ 989.893417][ T32] ? load_module (kernel/module/main.c:2758 kernel/module/main.c:2945)
[ 989.894035][ T32] load_module (kernel/module/main.c:2965)
[ 989.894644][ T32] ? post_relocation (kernel/module/main.c:2829)
[ 989.895343][ T32] ? __x64_sys_fspick (fs/kernel_read_file.c:38)
[ 989.896301][ T32] ? __cond_resched (kernel/sched/core.c:8533)
[ 989.897515][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099)
[ 989.903046][ T32] __do_sys_finit_module (kernel/module/main.c:3099)
[ 989.904404][ T32] ? __ia32_sys_init_module (kernel/module/main.c:3061)
[ 989.905751][ T32] ? randomize_page (mm/util.c:533)
[ 989.906912][ T32] ? ksys_mmap_pgoff (mm/mmap.c:1445)
[ 989.907570][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 989.908151][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 989.908891][ T32] RIP: 0033:0x7feb8112a9b9
[ 989.909479][ T32] RSP: 002b:00007ffd74170a98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 989.910515][ T32] RAX: ffffffffffffffda RBX: 000055ac05343d30 RCX: 00007feb8112a9b9
[ 989.911580][ T32] RDX: 0000000000000000 RSI: 000055ac05277260 RDI: 0000000000000003
[ 989.912567][ T32] RBP: 0000000000060000 R08: 0000000000000000 R09: 000055ac053458b0
[ 989.913550][ T32] R10: 0000000000000003 R11: 0000000000000246 R12: 000055ac05277260
[ 989.914742][ T32] R13: 0000000000000000 R14: 000055ac05343cb0 R15: 000055ac05343d30
[ 989.915789][ T32] </TASK>
[ 989.916463][ T32] INFO: task modprobe:91 blocked for more than 491 seconds.
[ 989.917554][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 989.918474][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.920351][ T32] task:modprobe state:D stack:0 pid:91 ppid:1 flags:0x00004002
[ 989.922592][ T32] Call Trace:
[ 989.923606][ T32] <TASK>
[ 989.924503][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 989.925717][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 989.926964][ T32] ? osq_unlock (kernel/locking/osq_lock.c:22 kernel/locking/osq_lock.c:210)
[ 989.927574][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 989.928128][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 989.928826][ T32] rwsem_down_write_slowpath (arch/x86/include/asm/current.h:41 kernel/locking/rwsem.c:1180)
[ 989.929533][ T32] ? down_timeout (kernel/locking/rwsem.c:1108)
[ 989.930115][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 989.930743][ T32] ? __kernfs_new_node (fs/kernfs/dir.c:651)
[ 989.931621][ T32] down_write (kernel/locking/rwsem.c:1306 kernel/locking/rwsem.c:1315 kernel/locking/rwsem.c:1574)
[ 989.932705][ T32] ? rwsem_down_write_slowpath (kernel/locking/rwsem.c:1571)
[ 989.933976][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 989.935249][ T32] ? notifier_call_chain (kernel/notifier.c:95)
[ 989.936333][ T32] ? blocking_notifier_call_chain_robust (kernel/notifier.c:129 kernel/notifier.c:353 kernel/notifier.c:341)
[ 989.937360][ T32] ? load_module (include/linux/notifier.h:209 kernel/module/main.c:2764 kernel/module/main.c:2945)
[ 989.938087][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099)
[ 989.938892][ T32] ? do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 989.939627][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 989.940609][ T32] kernfs_add_one (include/linux/kernfs.h:391 fs/kernfs/dir.c:754)
[ 989.941241][ T32] kernfs_create_dir_ns (fs/kernfs/dir.c:1044)
[ 989.942082][ T32] sysfs_create_dir_ns (fs/sysfs/dir.c:61)
[ 989.942885][ T32] ? sysfs_create_mount_point (fs/sysfs/dir.c:41)
[ 989.943803][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 989.944568][ T32] ? __kmalloc_node_track_caller (include/linux/kasan.h:196 mm/slab_common.c:966 mm/slab_common.c:986)
[ 989.945448][ T32] kobject_add_internal (lib/kobject.c:65 lib/kobject.c:233)
[ 989.946551][ T32] kobject_init_and_add (lib/kobject.c:368 lib/kobject.c:451)
[ 989.947309][ T32] ? kobject_create_and_add (lib/kobject.c:444)
[ 989.948025][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 989.948665][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153)
[ 989.949347][ T32] ? ddebug_module_notify (lib/dynamic_debug.c:1344 lib/dynamic_debug.c:1336)
[ 989.950021][ T32] mod_sysfs_setup (kernel/module/sysfs.c:361 kernel/module/sysfs.c:377)
[ 989.950943][ T32] ? module_add_modinfo_attrs (kernel/module/sysfs.c:374)
[ 989.952014][ T32] ? atomic_notifier_call_chain (kernel/notifier.c:343)
[ 989.953224][ T32] ? klp_module_coming (kernel/livepatch/core.c:1296)
[ 989.954395][ T32] ? load_module (kernel/module/main.c:2758 kernel/module/main.c:2945)
[ 989.955507][ T32] load_module (kernel/module/main.c:2965)
[ 989.956552][ T32] ? post_relocation (kernel/module/main.c:2829)
[ 989.957756][ T32] ? __x64_sys_fspick (fs/kernel_read_file.c:38)
[ 989.958858][ T32] ? __cond_resched (kernel/sched/core.c:8533)
[ 989.959756][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099)
[ 989.961072][ T32] __do_sys_finit_module (kernel/module/main.c:3099)
[ 989.962177][ T32] ? __ia32_sys_init_module (kernel/module/main.c:3061)
[ 989.963144][ T32] ? randomize_page (mm/util.c:533)
[ 989.963862][ T32] ? ksys_mmap_pgoff (mm/mmap.c:1445)
[ 989.964664][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 989.965454][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 989.966270][ T32] RIP: 0033:0x7f1d905f79b9
[ 989.967030][ T32] RSP: 002b:00007ffc9133bbd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 989.968331][ T32] RAX: ffffffffffffffda RBX: 00005651b1b1ae20 RCX: 00007f1d905f79b9
[ 989.969548][ T32] RDX: 0000000000000000 RSI: 00005651b1622260 RDI: 0000000000000003
[ 989.970830][ T32] RBP: 0000000000060000 R08: 0000000000000000 R09: 00005651b1b1c8b0
[ 989.972058][ T32] R10: 0000000000000003 R11: 0000000000000246 R12: 00005651b1622260
[ 989.973341][ T32] R13: 0000000000000000 R14: 00005651b1b1af50 R15: 00005651b1b1ae20
[ 989.974561][ T32] </TASK>
[ 989.975037][ T32] INFO: task systemd-journal:92 blocked for more than 491 seconds.
[ 989.976285][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 989.977338][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.979000][ T32] task:systemd-journal state:D stack:0 pid:92 ppid:1 flags:0x00000000
[ 989.981119][ T32] Call Trace:
[ 989.982020][ T32] <TASK>
[ 989.982888][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 989.983793][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 989.984786][ T32] ? unwind_next_frame (arch/x86/kernel/unwind_orc.c:378 arch/x86/kernel/unwind_orc.c:620)
[ 989.985459][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:24)
[ 989.986085][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414)
[ 989.986828][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 989.987663][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 989.988361][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073)
[ 989.989399][ T32] ? down_write_killable (kernel/locking/rwsem.c:997)
[ 989.990086][ T32] ? file_fdatawait_range (mm/filemap.c:3485)
[ 989.990921][ T32] down_read (kernel/locking/rwsem.c:1518)
[ 989.991817][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518)
[ 989.993060][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86)
[ 989.994335][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265)
[ 989.995528][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135)
[ 989.996846][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651)
[ 989.997996][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294)
[ 990.000155][ T32] walk_component (fs/namei.c:1994)
[ 990.001367][ T32] link_path_walk+0x533/0xa00
[ 990.002933][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243)
[ 990.004345][ T32] path_lookupat (fs/namei.c:2248 (discriminator 2) fs/namei.c:2478 (discriminator 2))
[ 990.005493][ T32] filename_lookup (fs/namei.c:2509)
[ 990.006694][ T32] ? may_linkat (fs/namei.c:2502)
[ 990.007891][ T32] ? strncpy_from_user (arch/x86/include/asm/uaccess.h:605 lib/strncpy_from_user.c:138)
[ 990.009149][ T32] user_path_at_empty (fs/namei.c:2909)
[ 990.010387][ T32] user_statfs (include/linux/namei.h:57 fs/statfs.c:103)
[ 990.011522][ T32] ? __ia32_sys_ustat (fs/statfs.c:98)
[ 990.012771][ T32] __do_sys_statfs (fs/statfs.c:196)
[ 990.013989][ T32] ? user_statfs (fs/statfs.c:193)
[ 990.015146][ T32] ? up_read (arch/x86/include/asm/atomic64_64.h:160 include/linux/atomic/atomic-long.h:71 include/linux/atomic/atomic-instrumented.h:1362 kernel/locking/rwsem.c:1347 kernel/locking/rwsem.c:1616)
[ 990.015775][ T32] ? syscall_trace_enter+0x96/0x190
[ 990.016557][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 990.017147][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 990.017918][ T32] RIP: 0033:0x7fd5039e98c7
[ 990.018530][ T32] RSP: 002b:00007ffd5f26e0b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
[ 990.019615][ T32] RAX: ffffffffffffffda RBX: 00007ffd5f26e130 RCX: 00007fd5039e98c7
[ 990.020612][ T32] RDX: 00007fd503aea1a4 RSI: 00007ffd5f26e130 RDI: 00007fd503ae0843
[ 990.021627][ T32] RBP: 00007fd503ae0843 R08: 0000000000000000 R09: 0000000000000000
[ 990.022622][ T32] R10: 00000000000002b0 R11: 0000000000000246 R12: 00007ffd5f26e2b8
[ 990.023667][ T32] R13: 00007ffd5f26e2c8 R14: 00007fd503ae8610 R15: 0000000000000000
[ 990.024655][ T32] </TASK>
[ 990.025154][ T32] INFO: task systemd-modules:93 blocked for more than 491 seconds.
[ 990.027363][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 990.028884][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 990.030964][ T32] task:systemd-modules state:D stack:0 pid:93 ppid:1 flags:0x00000000
[ 990.032126][ T32] Call Trace:
[ 990.032627][ T32] <TASK>
[ 990.033062][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 990.033683][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 990.034550][ T32] ? unwind_next_frame (arch/x86/kernel/unwind_orc.c:378 arch/x86/kernel/unwind_orc.c:620)
[ 990.035252][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:24)
[ 990.035871][ T32] ? is_bpf_text_address (arch/x86/include/asm/preempt.h:85 include/linux/rcupdate.h:99 include/linux/rcupdate.h:805 kernel/bpf/core.c:721)
[ 990.036528][ T32] ? kernel_text_address (kernel/extable.c:97 kernel/extable.c:94)
[ 990.037170][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414)
[ 990.038056][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 990.038750][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 990.039811][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073)
[ 990.040514][ T32] ? down_write_killable (kernel/locking/rwsem.c:997)
[ 990.041320][ T32] ? file_fdatawait_range (mm/filemap.c:3485)
[ 990.042565][ T32] down_read (kernel/locking/rwsem.c:1518)
[ 990.043578][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518)
[ 990.044897][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86)
[ 990.046130][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265)
[ 990.053684][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135)
[ 990.055053][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651)
[ 990.056152][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294)
[ 990.057463][ T32] walk_component (fs/namei.c:1994)
[ 990.058592][ T32] link_path_walk+0x533/0xa00
[ 990.060119][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243)
[ 990.061518][ T32] path_lookupat (fs/namei.c:2248 (discriminator 2) fs/namei.c:2478 (discriminator 2))
[ 990.062658][ T32] filename_lookup (fs/namei.c:2509)
[ 990.063775][ T32] ? may_linkat (fs/namei.c:2502)
[ 990.064820][ T32] ? strncpy_from_user (arch/x86/include/asm/uaccess.h:605 lib/strncpy_from_user.c:138)
[ 990.066006][ T32] user_path_at_empty (fs/namei.c:2909)
[ 990.067157][ T32] user_statfs (include/linux/namei.h:57 fs/statfs.c:103)
[ 990.068186][ T32] ? __ia32_sys_ustat (fs/statfs.c:98)
[ 990.069355][ T32] __do_sys_statfs (fs/statfs.c:196)
[ 990.070342][ T32] ? user_statfs (fs/statfs.c:193)
[ 990.071166][ T32] ? mm_account_fault (arch/x86/include/asm/irqflags.h:134 include/linux/memcontrol.h:1079 include/linux/memcontrol.h:1111 include/linux/memcontrol.h:1100 mm/memory.c:5123)
[ 990.071937][ T32] ? handle_mm_fault (mm/memory.c:5262)
[ 990.072580][ T32] ? up_read (arch/x86/include/asm/atomic64_64.h:160 include/linux/atomic/atomic-long.h:71 include/linux/atomic/atomic-instrumented.h:1362 kernel/locking/rwsem.c:1347 kernel/locking/rwsem.c:1616)
[ 990.073128][ T32] ? do_user_addr_fault (include/linux/mmap_lock.h:170 arch/x86/mm/fault.c:1468)
[ 990.073798][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 990.074387][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 990.075287][ T32] RIP: 0033:0x7f1b978518c7
[ 990.075911][ T32] RSP: 002b:00007ffdb557fc48 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
[ 990.077454][ T32] RAX: ffffffffffffffda RBX: 00007ffdb557fcc0 RCX: 00007f1b978518c7
[ 990.078536][ T32] RDX: 00007f1b974d11a4 RSI: 00007ffdb557fcc0 RDI: 00007f1b974c7843
[ 990.079636][ T32] RBP: 00007f1b974c7843 R08: 0000000000000000 R09: 9fb8220300000000
[ 990.080743][ T32] R10: 00000000478bfbff R11: 0000000000000246 R12: 00007ffdb557fe48
[ 990.081743][ T32] R13: 00007ffdb557fe58 R14: 00007f1b974cf610 R15: 0000000000000000
[ 990.082740][ T32] </TASK>
[ 990.083412][ T32] INFO: task systemd-remount:94 blocked for more than 491 seconds.
[ 990.085248][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 990.086674][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 990.089254][ T32] task:systemd-remount state:D stack:0 pid:94 ppid:1 flags:0x00000000
[ 990.091522][ T32] Call Trace:
[ 990.092433][ T32] <TASK>
[ 990.093316][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 990.094513][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 990.095872][ T32] ? unwind_next_frame (arch/x86/kernel/unwind_orc.c:378 arch/x86/kernel/unwind_orc.c:620)
[ 990.097243][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:24)
[ 990.098489][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414)
[ 990.099700][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 990.100802][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 990.102163][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073)
[ 990.103566][ T32] ? down_write_killable (kernel/locking/rwsem.c:997)
[ 990.104932][ T32] ? file_fdatawait_range (mm/filemap.c:3485)
[ 990.106279][ T32] down_read (kernel/locking/rwsem.c:1518)
[ 990.107433][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518)
[ 990.108819][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86)
[ 990.110093][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265)
[ 990.111095][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135)
[ 990.111862][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651)
[ 990.112824][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294)
[ 990.114078][ T32] walk_component (fs/namei.c:1994)
[ 990.114725][ T32] link_path_walk+0x533/0xa00
[ 990.116148][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243)
[ 990.117495][ T32] path_lookupat (fs/namei.c:2248 (discriminator 2) fs/namei.c:2478 (discriminator 2))
[ 990.118168][ T32] filename_lookup (fs/namei.c:2509)
[ 990.118831][ T32] ? may_linkat (fs/namei.c:2502)
[ 990.119593][ T32] ? strncpy_from_user (arch/x86/include/asm/uaccess.h:605 lib/strncpy_from_user.c:138)
[ 990.120820][ T32] user_path_at_empty (fs/namei.c:2909)
[ 990.121746][ T32] user_statfs (include/linux/namei.h:57 fs/statfs.c:103)
[ 990.122653][ T32] ? __ia32_sys_ustat (fs/statfs.c:98)
[ 990.123499][ T32] __do_sys_statfs (fs/statfs.c:196)
[ 990.124588][ T32] ? user_statfs (fs/statfs.c:193)
[ 990.125742][ T32] ? mm_account_fault (arch/x86/include/asm/irqflags.h:134 include/linux/memcontrol.h:1079 include/linux/memcontrol.h:1111 include/linux/memcontrol.h:1100 mm/memory.c:5123)
[ 990.126897][ T32] ? handle_mm_fault (mm/memory.c:5262)
[ 990.128044][ T32] ? up_read (arch/x86/include/asm/atomic64_64.h:160 include/linux/atomic/atomic-long.h:71 include/linux/atomic/atomic-instrumented.h:1362 kernel/locking/rwsem.c:1347 kernel/locking/rwsem.c:1616)
[ 990.129025][ T32] ? do_user_addr_fault (include/linux/mmap_lock.h:170 arch/x86/mm/fault.c:1468)
[ 990.130186][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 990.131288][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 990.132597][ T32] RIP: 0033:0x7f2eda3488c7
[ 990.133566][ T32] RSP: 002b:00007ffce4e63338 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
[ 990.135381][ T32] RAX: ffffffffffffffda RBX: 00007ffce4e633b0 RCX: 00007f2eda3488c7
[ 990.137068][ T32] RDX: 00007f2ed9fac1a4 RSI: 00007ffce4e633b0 RDI: 00007f2ed9fa2843
[ 990.138642][ T32] RBP: 00007f2ed9fa2843 R08: 0000000000000000 R09: 0000000000000000
[ 990.140356][ T32] R10: 00000000000002b0 R11: 0000000000000246 R12: 00007ffce4e63538
[ 990.142047][ T32] R13: 00007ffce4e63548 R14: 00007f2ed9faa610 R15: 0000000000000000
[ 990.143852][ T32] </TASK>
[ 990.144662][ T32] INFO: task udevadm:95 blocked for more than 491 seconds.
[ 990.146198][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 990.147520][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 990.149377][ T32] task:udevadm state:D stack:0 pid:95 ppid:1 flags:0x00000000
[ 990.151330][ T32] Call Trace:
[ 990.152142][ T32] <TASK>
[ 990.152910][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 990.153969][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 990.155177][ T32] ? kernel_text_address (kernel/extable.c:97 kernel/extable.c:94)
[ 990.156388][ T32] ? unwind_next_frame (arch/x86/kernel/unwind_orc.c:378 arch/x86/kernel/unwind_orc.c:620)
[ 990.157575][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:24)
[ 990.158682][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414)
[ 990.159875][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 990.160867][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 990.162097][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073)
[ 990.163373][ T32] ? down_write_killable (kernel/locking/rwsem.c:997)
[ 990.164566][ T32] ? file_fdatawait_range (mm/filemap.c:3485)
[ 990.165743][ T32] down_read (kernel/locking/rwsem.c:1518)
[ 990.166742][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518)
[ 990.168036][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86)
[ 990.169190][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265)
[ 990.170276][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135)
[ 990.171430][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651)
[ 990.172459][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294)
[ 990.173637][ T32] walk_component (fs/namei.c:1994)
[ 990.174710][ T32] link_path_walk+0x533/0xa00
[ 990.176103][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243)
[ 990.177383][ T32] path_lookupat (fs/namei.c:2248 (discriminator 2) fs/namei.c:2478 (discriminator 2))
[ 990.178528][ T32] filename_lookup (fs/namei.c:2509)
[ 990.179749][ T32] ? may_linkat (fs/namei.c:2502)
[ 990.180834][ T32] ? strncpy_from_user (arch/x86/include/asm/uaccess.h:605 lib/strncpy_from_user.c:138)
[ 990.181998][ T32] user_path_at_empty (fs/namei.c:2909)
[ 990.183148][ T32] user_statfs (include/linux/namei.h:57 fs/statfs.c:103)
[ 990.184197][ T32] ? __ia32_sys_ustat (fs/statfs.c:98)
[ 990.185340][ T32] __do_sys_statfs (fs/statfs.c:196)
[ 990.186430][ T32] ? user_statfs (fs/statfs.c:193)
[ 990.187538][ T32] ? mm_account_fault (arch/x86/include/asm/irqflags.h:134 include/linux/memcontrol.h:1079 include/linux/memcontrol.h:1111 include/linux/memcontrol.h:1100 mm/memory.c:5123)
[ 990.188691][ T32] ? handle_mm_fault (mm/memory.c:5262)
[ 990.189867][ T32] ? up_read (arch/x86/include/asm/atomic64_64.h:160 include/linux/atomic/atomic-long.h:71 include/linux/atomic/atomic-instrumented.h:1362 kernel/locking/rwsem.c:1347 kernel/locking/rwsem.c:1616)
[ 990.190867][ T32] ? do_user_addr_fault (include/linux/mmap_lock.h:170 arch/x86/mm/fault.c:1468)
[ 990.192059][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 990.193125][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 990.194455][ T32] RIP: 0033:0x7f7e23ebf8c7
[ 990.195521][ T32] RSP: 002b:00007ffe9f551448 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
[ 990.197388][ T32] RAX: ffffffffffffffda RBX: 00007ffe9f5514c0 RCX: 00007f7e23ebf8c7
[ 990.199194][ T32] RDX: 00007f7e23fe21a4 RSI: 00007ffe9f5514c0 RDI: 00007f7e23fd8843
[ 990.201122][ T32] RBP: 00007f7e23fd8843 R08: 0000000000000000 R09: 9fb8220300000000
[ 990.207119][ T32] R10: 00000000478bfbff R11: 0000000000000246 R12: 00007ffe9f551648
[ 990.208948][ T32] R13: 00007ffe9f551670 R14: 00007f7e23fe0610 R15: 0000000000000000
[ 990.210742][ T32] </TASK>
[ 1481.315358][ T32] INFO: task systemd:1 blocked for more than 983 seconds.
[ 1481.316989][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 1481.318380][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1481.320287][ T32] task:systemd state:D stack:0 pid:1 ppid:0 flags:0x00000000
[ 1481.322300][ T32] Call Trace:
[ 1481.323089][ T32] <TASK>
[ 1481.323722][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 1481.324790][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 1481.325948][ T32] ? is_bpf_text_address (arch/x86/include/asm/preempt.h:85 include/linux/rcupdate.h:99 include/linux/rcupdate.h:805 kernel/bpf/core.c:721)
[ 1481.327112][ T32] ? __kernel_text_address (kernel/extable.c:79)
[ 1481.332555][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:26)
[ 1481.333657][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414)
[ 1481.334800][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 1481.335755][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 1481.337010][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073)
[ 1481.338251][ T32] ? filter_irq_stacks (kernel/stacktrace.c:114)
[ 1481.339367][ T32] ? down_write_killable (kernel/locking/rwsem.c:997)
[ 1481.340483][ T32] ? kasan_save_stack (mm/kasan/common.c:46)
[ 1481.341479][ T32] ? kasan_set_track (mm/kasan/common.c:52)
[ 1481.342515][ T32] ? do_filp_open (fs/namei.c:601 fs/namei.c:612 fs/namei.c:3817)
[ 1481.343547][ T32] ? do_sys_openat2 (fs/open.c:1356)
[ 1481.344644][ T32] ? __x64_sys_openat (fs/open.c:1383)
[ 1481.345757][ T32] ? do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 1481.346788][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 1481.348135][ T32] down_read (kernel/locking/rwsem.c:1518)
[ 1481.349175][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518)
[ 1481.350421][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86)
[ 1481.357275][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265)
[ 1481.358309][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135)
[ 1481.359461][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651)
[ 1481.360441][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294)
[ 1481.361528][ T32] walk_component (fs/namei.c:1994)
[ 1481.362501][ T32] link_path_walk+0x533/0xa00
[ 1481.363800][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243)
[ 1481.365048][ T32] ? __mutex_init (arch/x86/include/asm/atomic.h:41 include/linux/atomic/atomic-instrumented.h:42 include/linux/osq_lock.h:30 kernel/locking/mutex.c:52)
[ 1481.366100][ T32] ? __alloc_file (fs/file_table.c:154)
[ 1481.367082][ T32] path_openat (fs/namei.c:2250 (discriminator 2) fs/namei.c:3787 (discriminator 2))
[ 1481.368064][ T32] ? vfs_tmpfile_open (fs/namei.c:3773)
[ 1481.369074][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 1481.370313][ T32] do_filp_open (fs/namei.c:3818)
[ 1481.371266][ T32] ? __update_load_avg_se (kernel/sched/pelt.c:118 kernel/sched/pelt.c:226 kernel/sched/pelt.c:308)
[ 1481.372368][ T32] ? may_open_dev (fs/namei.c:3812)
[ 1481.373365][ T32] ? update_load_avg (kernel/sched/fair.c:3920 kernel/sched/fair.c:4255)
[ 1481.374450][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 1481.375378][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153)
[ 1481.376371][ T32] ? alloc_fd (arch/x86/include/asm/bitops.h:68 include/asm-generic/bitops/instrumented-non-atomic.h:29 fs/file.c:251 fs/file.c:540)
[ 1481.377250][ T32] do_sys_openat2 (fs/open.c:1356)
[ 1481.378142][ T32] ? build_open_flags (fs/open.c:1342)
[ 1481.379113][ T32] __x64_sys_openat (fs/open.c:1383)
[ 1481.380196][ T32] ? __ia32_compat_sys_open (fs/open.c:1383)
[ 1481.381372][ T32] ? schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 1481.382397][ T32] ? switch_fpu_return (arch/x86/include/asm/bitops.h:75 include/asm-generic/bitops/instrumented-atomic.h:42 include/linux/thread_info.h:94 arch/x86/kernel/fpu/context.h:80 arch/x86/kernel/fpu/core.c:752)
[ 1481.383565][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 1481.384625][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 1481.385922][ T32] RIP: 0033:0x7f5c316f8be7
[ 1481.386955][ T32] RSP: 002b:00007ffc354a1f10 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
[ 1481.388659][ T32] RAX: ffffffffffffffda RBX: 0000562309e04d60 RCX: 00007f5c316f8be7
[ 1481.390340][ T32] RDX: 0000000000080000 RSI: 0000562309eb1320 RDI: 00000000ffffff9c
[ 1481.396264][ T32] RBP: 0000562309eb1320 R08: 0000000000000008 R09: 0000000000000001
[ 1481.397989][ T32] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000080000
[ 1481.399627][ T32] R13: 0000562309e04d60 R14: 0000000000000001 R15: 00007ffc354a2320
[ 1481.401415][ T32] </TASK>
[ 1481.402318][ T32] INFO: task modprobe:90 blocked for more than 983 seconds.
[ 1481.403796][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 1481.405100][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1481.406876][ T32] task:modprobe state:D stack:0 pid:90 ppid:1 flags:0x00004002
[ 1481.408746][ T32] Call Trace:
[ 1481.409491][ T32] <TASK>
[ 1481.410226][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 1481.411131][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 1481.412410][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 1481.413686][ T32] ? idr_get_free (arch/x86/include/asm/bitops.h:228 arch/x86/include/asm/bitops.h:240 include/asm-generic/bitops/instrumented-non-atomic.h:142 lib/radix-tree.c:113 lib/radix-tree.c:1518)
[ 1481.414741][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414)
[ 1481.415892][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 1481.416818][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 1481.417989][ T32] rwsem_down_write_slowpath (arch/x86/include/asm/current.h:41 kernel/locking/rwsem.c:1180)
[ 1481.419239][ T32] ? down_timeout (kernel/locking/rwsem.c:1108)
[ 1481.420298][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 1481.421353][ T32] ? __kernfs_new_node (fs/kernfs/dir.c:651)
[ 1481.422530][ T32] ? stack_trace_save (kernel/stacktrace.c:123)
[ 1481.423640][ T32] down_write (kernel/locking/rwsem.c:1306 kernel/locking/rwsem.c:1315 kernel/locking/rwsem.c:1574)
[ 1481.424626][ T32] ? rwsem_down_write_slowpath (kernel/locking/rwsem.c:1571)
[ 1481.425959][ T32] ? kasan_save_stack (mm/kasan/common.c:47)
[ 1481.427073][ T32] ? kasan_save_stack (mm/kasan/common.c:46)
[ 1481.428236][ T32] kernfs_add_one (include/linux/kernfs.h:391 fs/kernfs/dir.c:754)
[ 1481.429310][ T32] ? pcpu_chunk_refresh_hint (mm/percpu-internal.h:114 (discriminator 3) mm/percpu.c:762 (discriminator 3))
[ 1481.430476][ T32] kernfs_create_dir_ns (fs/kernfs/dir.c:1044)
[ 1481.431645][ T32] sysfs_create_dir_ns (fs/sysfs/dir.c:61)
[ 1481.432719][ T32] ? sysfs_create_mount_point (fs/sysfs/dir.c:41)
[ 1481.433765][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 1481.434806][ T32] ? __kmalloc_node_track_caller (include/linux/kasan.h:196 mm/slab_common.c:966 mm/slab_common.c:986)
[ 1481.436081][ T32] kobject_add_internal (lib/kobject.c:65 lib/kobject.c:233)
[ 1481.437226][ T32] kobject_init_and_add (lib/kobject.c:368 lib/kobject.c:451)
[ 1481.438323][ T32] ? kobject_create_and_add (lib/kobject.c:444)
[ 1481.439458][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 1481.440515][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153)
[ 1481.441649][ T32] ? srcu_module_notify (kernel/rcu/srcutree.c:1921 kernel/rcu/srcutree.c:1954)
[ 1481.442760][ T32] ? tracepoint_module_notify (kernel/tracepoint.c:664 kernel/tracepoint.c:709 kernel/tracepoint.c:701)
[ 1481.443936][ T32] mod_sysfs_setup (kernel/module/sysfs.c:361 kernel/module/sysfs.c:377)
[ 1481.444989][ T32] ? module_add_modinfo_attrs (kernel/module/sysfs.c:374)
[ 1481.446267][ T32] ? atomic_notifier_call_chain (kernel/notifier.c:343)
[ 1481.447469][ T32] ? klp_module_coming (kernel/livepatch/core.c:1296)
[ 1481.448606][ T32] ? load_module (kernel/module/main.c:2758 kernel/module/main.c:2945)
[ 1481.449662][ T32] load_module (kernel/module/main.c:2965)
[ 1481.450708][ T32] ? post_relocation (kernel/module/main.c:2829)
[ 1481.451842][ T32] ? __x64_sys_fspick (fs/kernel_read_file.c:38)
[ 1481.452990][ T32] ? __cond_resched (kernel/sched/core.c:8533)
[ 1481.454080][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099)
[ 1481.455293][ T32] __do_sys_finit_module (kernel/module/main.c:3099)
[ 1481.456528][ T32] ? __ia32_sys_init_module (kernel/module/main.c:3061)
[ 1481.457660][ T32] ? randomize_page (mm/util.c:533)
[ 1481.458766][ T32] ? ksys_mmap_pgoff (mm/mmap.c:1445)
[ 1481.459899][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 1481.460801][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 1481.462053][ T32] RIP: 0033:0x7feb8112a9b9
[ 1481.463109][ T32] RSP: 002b:00007ffd74170a98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 1481.465024][ T32] RAX: ffffffffffffffda RBX: 000055ac05343d30 RCX: 00007feb8112a9b9
[ 1481.466632][ T32] RDX: 0000000000000000 RSI: 000055ac05277260 RDI: 0000000000000003
[ 1481.468414][ T32] RBP: 0000000000060000 R08: 0000000000000000 R09: 000055ac053458b0
[ 1481.470222][ T32] R10: 0000000000000003 R11: 0000000000000246 R12: 000055ac05277260
[ 1481.471905][ T32] R13: 0000000000000000 R14: 000055ac05343cb0 R15: 000055ac05343d30
[ 1481.473600][ T32] </TASK>
[ 1481.474351][ T32] INFO: task modprobe:91 blocked for more than 983 seconds.
[ 1481.475846][ T32] Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 1481.477177][ T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1481.479050][ T32] task:modprobe state:D stack:0 pid:91 ppid:1 flags:0x00004002
[ 1481.480969][ T32] Call Trace:
[ 1481.481794][ T32] <TASK>
[ 1481.482535][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669)
[ 1481.483542][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551)
[ 1481.484706][ T32] ? osq_unlock (kernel/locking/osq_lock.c:22 kernel/locking/osq_lock.c:210)
[ 1481.485728][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1))
[ 1481.486675][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805)
[ 1481.487928][ T32] rwsem_down_write_slowpath (arch/x86/include/asm/current.h:41 kernel/locking/rwsem.c:1180)
[ 1481.489130][ T32] ? down_timeout (kernel/locking/rwsem.c:1108)
[ 1481.490248][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 1481.491354][ T32] ? __kernfs_new_node (fs/kernfs/dir.c:651)
[ 1481.492438][ T32] down_write (kernel/locking/rwsem.c:1306 kernel/locking/rwsem.c:1315 kernel/locking/rwsem.c:1574)
[ 1481.493342][ T32] ? rwsem_down_write_slowpath (kernel/locking/rwsem.c:1571)
[ 1481.494468][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 1481.495663][ T32] ? notifier_call_chain (kernel/notifier.c:95)
[ 1481.496756][ T32] ? blocking_notifier_call_chain_robust (kernel/notifier.c:129 kernel/notifier.c:353 kernel/notifier.c:341)
[ 1481.498045][ T32] ? load_module (include/linux/notifier.h:209 kernel/module/main.c:2764 kernel/module/main.c:2945)
[ 1481.499131][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099)
[ 1481.500344][ T32] ? do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 1481.501394][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 1481.502720][ T32] kernfs_add_one (include/linux/kernfs.h:391 fs/kernfs/dir.c:754)
[ 1481.503801][ T32] kernfs_create_dir_ns (fs/kernfs/dir.c:1044)
[ 1481.504981][ T32] sysfs_create_dir_ns (fs/sysfs/dir.c:61)
[ 1481.505997][ T32] ? sysfs_create_mount_point (fs/sysfs/dir.c:41)
[ 1481.507197][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 1481.509314][ T32] ? __kmalloc_node_track_caller (include/linux/kasan.h:196 mm/slab_common.c:966 mm/slab_common.c:986)
[ 1481.510619][ T32] kobject_add_internal (lib/kobject.c:65 lib/kobject.c:233)
[ 1481.511958][ T32] kobject_init_and_add (lib/kobject.c:368 lib/kobject.c:451)
[ 1481.513079][ T32] ? kobject_create_and_add (lib/kobject.c:444)
[ 1481.514276][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
[ 1481.515339][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153)
[ 1481.516463][ T32] ? ddebug_module_notify (lib/dynamic_debug.c:1344 lib/dynamic_debug.c:1336)
[ 1481.517587][ T32] mod_sysfs_setup (kernel/module/sysfs.c:361 kernel/module/sysfs.c:377)
[ 1481.518672][ T32] ? module_add_modinfo_attrs (kernel/module/sysfs.c:374)
[ 1481.519892][ T32] ? atomic_notifier_call_chain (kernel/notifier.c:343)
[ 1481.521125][ T32] ? klp_module_coming (kernel/livepatch/core.c:1296)
[ 1481.522271][ T32] ? load_module (kernel/module/main.c:2758 kernel/module/main.c:2945)
[ 1481.523331][ T32] load_module (kernel/module/main.c:2965)
[ 1481.524366][ T32] ? post_relocation (kernel/module/main.c:2829)
[ 1481.525461][ T32] ? __x64_sys_fspick (fs/kernel_read_file.c:38)
[ 1481.526548][ T32] ? __cond_resched (kernel/sched/core.c:8533)
[ 1481.527585][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099)
[ 1481.528682][ T32] __do_sys_finit_module (kernel/module/main.c:3099)
[ 1481.529764][ T32] ? __ia32_sys_init_module (kernel/module/main.c:3061)
[ 1481.530866][ T32] ? randomize_page (mm/util.c:533)
[ 1481.531860][ T32] ? ksys_mmap_pgoff (mm/mmap.c:1445)
[ 1481.532922][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80)
[ 1481.533948][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
[ 1481.535161][ T32] RIP: 0033:0x7f1d905f79b9
[ 1481.536136][ T32] RSP: 002b:00007ffc9133bbd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 1481.537935][ T32] RAX: ffffffffffffffda RBX: 00005651b1b1ae20 RCX: 00007f1d905f79b9
[ 1481.539611][ T32] RDX: 0000000000000000 RSI: 00005651b1622260 RDI: 0000000000000003
[ 1481.541313][ T32] RBP: 0000000000060000 R08: 0000000000000000 R09: 00005651b1b1c8b0
[ 1481.543097][ T32] R10: 0000000000000003 R11: 0000000000000246 R12: 00005651b1622260
[ 1481.544859][ T32] R13: 0000000000000000 R14: 00005651b1b1af50 R15: 00005651b1b1ae20
[ 1481.546692][ T32] </TASK>
[ 1481.547492][ T32] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
BUG: kernel hang in test stage
Kboot worker: lkp-worker18
Elapsed time: 2520
kvm=(
qemu-system-x86_64
-enable-kvm
-cpu SandyBridge
-kernel $kernel
-initrd initrd-vm-meta-147.cgz
-m 16384
-smp 2
-device e1000,netdev=net0
-netdev user,id=net0,hostfwd=tcp::32032-:22
-boot order=nc
-no-reboot
-device i6300esb
-watchdog-action debug
-rtc base=localtime
-serial stdio
-display none
-monitor null
)
append=(
ip=::::vm-meta-147::dhcp
root=/dev/ram0
RESULT_ROOT=/result/boot/1/vm-snb/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3-func/gcc-11/a551138c4b3b9fd7e74a55d6074a013bab6246db/3
BOOT_IMAGE=/pkg/linux/x86_64-rhel-8.3-func/gcc-11/a551138c4b3b9fd7e74a55d6074a013bab6246db/vmlinuz-6.3.0-12053-ga551138c4b3b
branch=linux-review/Dominique-Martinet/fs-split-off-vfs_getdents-function-of-getdents64-syscall/20230510-185542
job=/job-script
user=lkp
ARCH=x86_64
kconfig=x86_64-rhel-8.3-func
commit=a551138c4b3b9fd7e74a55d6074a013bab6246db
initcall_debug
nmi_watchdog=0
vmalloc=256M
initramfs_async=0
To reproduce:
# build kernel
cd linux
cp config-6.3.0-12053-ga551138c4b3b .config
make HOSTCC=gcc-11 CC=gcc-11 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-11 CC=gcc-11 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
On Wed, May 10, 2023 at 07:52:54PM +0900, Dominique Martinet wrote:
> This turns out to be very slightly faster than an extra call to
> getdents, but in practice it doesn't seem to be such an improvement as
> the trailing getdents will return almost immediately be absorbed by the
> scheduling noise in a find-like context (my ""server"" is too noisy to
> get proper benchmarks out, but results look slightly better with this in
> async mode, and almost identical in the NOWAIT path)
>
> If the user is waiting the end of a single directory though it might be
> worth it, so including the patch for comments.
> (in particular I'm not really happy that the flag has become in-out for
> vfs_getdents, especially when the getdents64 syscall does not use it,
> but I don't see much other way around it)
>
> If this approach is acceptable/wanted then this patch will be split down
> further (at least dir_context/vfs_getdents, kernfs, libfs, uring in four
> separate commits)
>
> Signed-off-by: Dominique Martinet <[email protected]>
> ---
> fs/internal.h | 2 +-
> fs/kernfs/dir.c | 1 +
> fs/libfs.c | 9 ++++++---
> fs/readdir.c | 10 ++++++----
> include/linux/fs.h | 2 ++
> include/uapi/linux/io_uring.h | 2 ++
> io_uring/fs.c | 8 ++++++--
> 7 files changed, 24 insertions(+), 10 deletions(-)
>
> diff --git a/fs/internal.h b/fs/internal.h
> index 0264b001d99a..0b1552c7a870 100644
> --- a/fs/internal.h
> +++ b/fs/internal.h
> @@ -267,4 +267,4 @@ void mnt_idmap_put(struct mnt_idmap *idmap);
> struct linux_dirent64;
>
> int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
> - unsigned int count, unsigned long flags);
> + unsigned int count, unsigned long *flags);
> diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c
> index 5a5b3e7881bf..53a6b4804c34 100644
> --- a/fs/kernfs/dir.c
> +++ b/fs/kernfs/dir.c
> @@ -1860,6 +1860,7 @@ static int kernfs_fop_readdir(struct file *file, struct dir_context *ctx)
> up_read(&root->kernfs_rwsem);
> file->private_data = NULL;
> ctx->pos = INT_MAX;
> + ctx->flags |= DIR_CONTEXT_F_EOD;
> return 0;
> }
>
> diff --git a/fs/libfs.c b/fs/libfs.c
> index a3c7e42d90a7..b2a95dadffbd 100644
> --- a/fs/libfs.c
> +++ b/fs/libfs.c
> @@ -208,10 +208,12 @@ int dcache_readdir(struct file *file, struct dir_context *ctx)
> p = &next->d_child;
> }
> spin_lock(&dentry->d_lock);
> - if (next)
> + if (next) {
> list_move_tail(&cursor->d_child, &next->d_child);
> - else
> + } else {
> list_del_init(&cursor->d_child);
> + ctx->flags |= DIR_CONTEXT_F_EOD;
> + }
> spin_unlock(&dentry->d_lock);
> dput(next);
>
> @@ -1347,7 +1349,8 @@ static loff_t empty_dir_llseek(struct file *file, loff_t offset, int whence)
>
> static int empty_dir_readdir(struct file *file, struct dir_context *ctx)
> {
> - dir_emit_dots(file, ctx);
> + if (dir_emit_dots(file, ctx))
> + ctx->flags |= DIR_CONTEXT_F_EOD;
> return 0;
> }
>
> diff --git a/fs/readdir.c b/fs/readdir.c
> index 1311b89d75e1..be75a2154b4f 100644
> --- a/fs/readdir.c
> +++ b/fs/readdir.c
> @@ -358,14 +358,14 @@ static bool filldir64(struct dir_context *ctx, const char *name, int namlen,
> * @file : pointer to file struct of directory
> * @dirent : pointer to user directory structure
> * @count : size of buffer
> - * @flags : additional dir_context flags
> + * @flags : pointer to additional dir_context flags
> */
> int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
> - unsigned int count, unsigned long flags)
> + unsigned int count, unsigned long *flags)
> {
> struct getdents_callback64 buf = {
> .ctx.actor = filldir64,
> - .ctx.flags = flags,
> + .ctx.flags = flags ? *flags : 0,
> .count = count,
> .current_dir = dirent
> };
> @@ -384,6 +384,8 @@ int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
> else
> error = count - buf.count;
> }
> + if (flags)
> + *flags = buf.ctx.flags;
> return error;
> }
>
> @@ -397,7 +399,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,
> if (!f.file)
> return -EBADF;
>
> - error = vfs_getdents(f.file, dirent, count, 0);
> + error = vfs_getdents(f.file, dirent, count, NULL);
>
> fdput_pos(f);
> return error;
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index f7de2b5ca38e..d1e31bccfb4f 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -1723,8 +1723,10 @@ struct dir_context {
> * flags for dir_context flags
> * DIR_CONTEXT_F_NOWAIT: Request non-blocking iterate
> * (requires file->f_mode & FMODE_NOWAIT)
> + * DIR_CONTEXT_F_EOD: Signal directory has been fully iterated, set by the fs
> */
> #define DIR_CONTEXT_F_NOWAIT 0x1
> +#define DIR_CONTEXT_F_EOD 0x2
>
> /*
> * These flags let !MMU mmap() govern direct device mapping vs immediate
> diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
> index 35d0de18d893..35877132027e 100644
> --- a/include/uapi/linux/io_uring.h
> +++ b/include/uapi/linux/io_uring.h
> @@ -381,11 +381,13 @@ struct io_uring_cqe {
> * IORING_CQE_F_SOCK_NONEMPTY If set, more data to read after socket recv
> * IORING_CQE_F_NOTIF Set for notification CQEs. Can be used to distinct
> * them from sends.
> + * IORING_CQE_F_EOF If set, file or directory has reached end of file.
> */
> #define IORING_CQE_F_BUFFER (1U << 0)
> #define IORING_CQE_F_MORE (1U << 1)
> #define IORING_CQE_F_SOCK_NONEMPTY (1U << 2)
> #define IORING_CQE_F_NOTIF (1U << 3)
> +#define IORING_CQE_F_EOF (1U << 4)
>
> enum {
> IORING_CQE_BUFFER_SHIFT = 16,
> diff --git a/io_uring/fs.c b/io_uring/fs.c
> index b15ec81c1ed2..f6222b0148ef 100644
> --- a/io_uring/fs.c
> +++ b/io_uring/fs.c
> @@ -322,6 +322,7 @@ int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
> {
> struct io_getdents *gd = io_kiocb_to_cmd(req, struct io_getdents);
> unsigned long getdents_flags = 0;
> + u32 cqe_flags = 0;
> int ret;
>
> if (issue_flags & IO_URING_F_NONBLOCK) {
> @@ -338,13 +339,16 @@ int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
> goto out;
> }
>
> - ret = vfs_getdents(req->file, gd->dirent, gd->count, getdents_flags);
> + ret = vfs_getdents(req->file, gd->dirent, gd->count, &getdents_flags);
I don't understand how synchronization and updating of f_pos works here.
For example, what happens if a concurrent seek happens on the fd while
io_uring is using vfs_getdents which calls into iterate_dir() and
updates f_pos?
On Wed, May 10, 2023 at 07:52:49PM +0900, Dominique Martinet wrote:
> This splits off the vfs_getdents function from the getdents64 system
> call.
> This will allow io_uring to call the vfs_getdents function.
>
> Co-authored-by: Stefan Roesch <[email protected]>
> Signed-off-by: Dominique Martinet <[email protected]>
> ---
> fs/internal.h | 8 ++++++++
> fs/readdir.c | 34 ++++++++++++++++++++++++++--------
> 2 files changed, 34 insertions(+), 8 deletions(-)
>
> diff --git a/fs/internal.h b/fs/internal.h
> index bd3b2810a36b..e8ca000e6613 100644
> --- a/fs/internal.h
> +++ b/fs/internal.h
> @@ -260,3 +260,11 @@ ssize_t __kernel_write_iter(struct file *file, struct iov_iter *from, loff_t *po
> struct mnt_idmap *alloc_mnt_idmap(struct user_namespace *mnt_userns);
> struct mnt_idmap *mnt_idmap_get(struct mnt_idmap *idmap);
> void mnt_idmap_put(struct mnt_idmap *idmap);
> +
> +/*
> + * fs/readdir.c
> + */
> +struct linux_dirent64;
> +
> +int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
> + unsigned int count);
> diff --git a/fs/readdir.c b/fs/readdir.c
> index 9c53edb60c03..ed0803d0011e 100644
> --- a/fs/readdir.c
> +++ b/fs/readdir.c
> @@ -21,6 +21,7 @@
> #include <linux/unistd.h>
> #include <linux/compat.h>
> #include <linux/uaccess.h>
> +#include "internal.h"
>
> #include <asm/unaligned.h>
>
> @@ -351,10 +352,16 @@ static bool filldir64(struct dir_context *ctx, const char *name, int namlen,
> return false;
> }
>
> -SYSCALL_DEFINE3(getdents64, unsigned int, fd,
> - struct linux_dirent64 __user *, dirent, unsigned int, count)
> +
> +/**
> + * vfs_getdents - getdents without fdget
> + * @file : pointer to file struct of directory
> + * @dirent : pointer to user directory structure
> + * @count : size of buffer
> + */
> +int vfs_getdents(struct file *file, struct linux_dirent64 __user *dirent,
> + unsigned int count)
> {
> - struct fd f;
> struct getdents_callback64 buf = {
> .ctx.actor = filldir64,
> .count = count,
> @@ -362,11 +369,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,
> };
> int error;
>
> - f = fdget_pos(fd);
> - if (!f.file)
> - return -EBADF;
> -
> - error = iterate_dir(f.file, &buf.ctx);
> + error = iterate_dir(file, &buf.ctx);
So afaict this isn't enough.
If you look into iterate_shared() you should see that it uses and
updates f_pos. But that can't work for io_uring and also isn't how
io_uring handles read and write. You probably need to use a local pos
similar to what io_uring does in rw.c for rw->kiocb.ki_pos. But in
contrast simply disallow any offsets for getdents completely. Thus not
relying on f_pos anywhere at all.
Christian Brauner wrote on Tue, May 23, 2023 at 04:30:14PM +0200:
> > index b15ec81c1ed2..f6222b0148ef 100644
> > --- a/io_uring/fs.c
> > +++ b/io_uring/fs.c
> > @@ -322,6 +322,7 @@ int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
> > {
> > struct io_getdents *gd = io_kiocb_to_cmd(req, struct io_getdents);
> > unsigned long getdents_flags = 0;
> > + u32 cqe_flags = 0;
> > int ret;
> >
> > if (issue_flags & IO_URING_F_NONBLOCK) {
> > @@ -338,13 +339,16 @@ int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
> > goto out;
> > }
> >
> > - ret = vfs_getdents(req->file, gd->dirent, gd->count, getdents_flags);
> > + ret = vfs_getdents(req->file, gd->dirent, gd->count, &getdents_flags);
>
> I don't understand how synchronization and updating of f_pos works here.
> For example, what happens if a concurrent seek happens on the fd while
> io_uring is using vfs_getdents which calls into iterate_dir() and
> updates f_pos?
I don't see how different that is from a user spawning two threads and
calling getdents64 + lseek or two getdents64 in parallel?
(or any two other users of iterate_dir)
As far as I understand you'll either get the old or new pos as
obtained/updated by iterate_dir()?
That iterate_dir probably ought to be using READ_ONCE/WRITE_ONCE or some
atomic read/update wrappers as the shared case only has a read lock
around these, but that's not a new problem; and for all I care
about I'm happy to let users shoot themselves in the foot.
(although I guess that with filesystems not validating the offset as
was pointed out in a previous version comment having non-atomic update
might be a security issue at some point on architectures that don't
guarantee atomic 64bit updates, but if someone manages to abuse it
it's already possible to abuse it with the good old syscalls, so I'd
rather leave that up to someone who understand how atomicity in the
kernel works better than me...)
--
Dominique Martinet | Asmadeus
Christian Brauner wrote on Tue, May 23, 2023 at 05:39:08PM +0200:
> > @@ -362,11 +369,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,
> > };
> > int error;
> >
> > - f = fdget_pos(fd);
> > - if (!f.file)
> > - return -EBADF;
> > -
> > - error = iterate_dir(f.file, &buf.ctx);
> > + error = iterate_dir(file, &buf.ctx);
>
> So afaict this isn't enough.
> If you look into iterate_shared() you should see that it uses and
> updates f_pos. But that can't work for io_uring and also isn't how
> io_uring handles read and write. You probably need to use a local pos
> similar to what io_uring does in rw.c for rw->kiocb.ki_pos. But in
> contrast simply disallow any offsets for getdents completely. Thus not
> relying on f_pos anywhere at all.
Using a private offset from the sqe was the previous implementation
discussed around here[1], and Al Viro pointed out that the iterate
filesystem implementations don't validate the offset makes sense as it's
either costly or for some filesystems downright impossible, so I went
into a don't let users modify it approach.
[1] https://lore.kernel.org/all/[email protected]/T/#m517583f23502f32b040c819d930359313b3db00c
I agree it's not how io_uring usually works -- it dislikes global
states -- but it works perfectly well as long as you don't have multiple
users on the same file, which the application can take care of.
Not having any offsets would work for small directories but make reading
large directories impossible so some sort of continuation is required,
which means we need to keep the offset around; I also suggested keeping
the offset in argument as the previous version but only allowing the
last known offset (... so ultimately still updating f_pos anyway as we
don't have anywhere else to store it) or 0, but if we're going to do
that it looks much simpler to me to expose the same API as getdents.
--
Dominique Martinet | Asmadeus
On Wed, May 24, 2023 at 06:13:57AM +0900, Dominique Martinet wrote:
> Christian Brauner wrote on Tue, May 23, 2023 at 05:39:08PM +0200:
> > > @@ -362,11 +369,7 @@ SYSCALL_DEFINE3(getdents64, unsigned int, fd,
> > > };
> > > int error;
> > >
> > > - f = fdget_pos(fd);
> > > - if (!f.file)
> > > - return -EBADF;
> > > -
> > > - error = iterate_dir(f.file, &buf.ctx);
> > > + error = iterate_dir(file, &buf.ctx);
> >
> > So afaict this isn't enough.
> > If you look into iterate_shared() you should see that it uses and
> > updates f_pos. But that can't work for io_uring and also isn't how
> > io_uring handles read and write. You probably need to use a local pos
> > similar to what io_uring does in rw.c for rw->kiocb.ki_pos. But in
> > contrast simply disallow any offsets for getdents completely. Thus not
> > relying on f_pos anywhere at all.
>
> Using a private offset from the sqe was the previous implementation
> discussed around here[1], and Al Viro pointed out that the iterate
> filesystem implementations don't validate the offset makes sense as it's
> either costly or for some filesystems downright impossible, so I went
> into a don't let users modify it approach.
>
> [1] https://lore.kernel.org/all/[email protected]/T/#m517583f23502f32b040c819d930359313b3db00c
>
>
> I agree it's not how io_uring usually works -- it dislikes global
> states -- but it works perfectly well as long as you don't have multiple
> users on the same file, which the application can take care of.
>
> Not having any offsets would work for small directories but make reading
> large directories impossible so some sort of continuation is required,
> which means we need to keep the offset around; I also suggested keeping
> the offset in argument as the previous version but only allowing the
> last known offset (... so ultimately still updating f_pos anyway as we
> don't have anywhere else to store it) or 0, but if we're going to do
> that it looks much simpler to me to expose the same API as getdents.
>
> --
> Dominique Martinet | Asmadeus
On Wed, May 24, 2023 at 06:05:06AM +0900, Dominique Martinet wrote:
> Christian Brauner wrote on Tue, May 23, 2023 at 04:30:14PM +0200:
> > > index b15ec81c1ed2..f6222b0148ef 100644
> > > --- a/io_uring/fs.c
> > > +++ b/io_uring/fs.c
> > > @@ -322,6 +322,7 @@ int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
> > > {
> > > struct io_getdents *gd = io_kiocb_to_cmd(req, struct io_getdents);
> > > unsigned long getdents_flags = 0;
> > > + u32 cqe_flags = 0;
> > > int ret;
> > >
> > > if (issue_flags & IO_URING_F_NONBLOCK) {
> > > @@ -338,13 +339,16 @@ int io_getdents(struct io_kiocb *req, unsigned int issue_flags)
> > > goto out;
> > > }
> > >
> > > - ret = vfs_getdents(req->file, gd->dirent, gd->count, getdents_flags);
> > > + ret = vfs_getdents(req->file, gd->dirent, gd->count, &getdents_flags);
> >
> > I don't understand how synchronization and updating of f_pos works here.
> > For example, what happens if a concurrent seek happens on the fd while
> > io_uring is using vfs_getdents which calls into iterate_dir() and
> > updates f_pos?
>
> I don't see how different that is from a user spawning two threads and
> calling getdents64 + lseek or two getdents64 in parallel?
> (or any two other users of iterate_dir)
>
> As far as I understand you'll either get the old or new pos as
> obtained/updated by iterate_dir()?
>
> That iterate_dir probably ought to be using READ_ONCE/WRITE_ONCE or some
> atomic read/update wrappers as the shared case only has a read lock
> around these, but that's not a new problem; and for all I care
> about I'm happy to let users shoot themselves in the foot.
> (although I guess that with filesystems not validating the offset as
> was pointed out in a previous version comment having non-atomic update
> might be a security issue at some point on architectures that don't
> guarantee atomic 64bit updates, but if someone manages to abuse it
> it's already possible to abuse it with the good old syscalls, so I'd
> rather leave that up to someone who understand how atomicity in the
> kernel works better than me...)
There's multiple issues here.
The main objection in [1] was to allow specifying an arbitrary offset
from userspace. What [3] did was to implement a pread() variant for
directories, i.e., pgetdents(). That can't work in principle/is
prohibitively complex. Which is what your series avoids by not allowing
any offsets to be specified.
However, there's still a problem here. Updates to f_pos happen under an
approriate lock to guarantee consistency of the position between calls
that move the cursor position. In the normal read-write path io_uring
doesn't concern itself with f_pos as it keeps local state in
kiocb->ki_pos.
But then it still does end up running into f_pos consistency problems
for read-write because it does allow operating on the current f_pos if
the offset if struct io_rw is set to -1.
In that case it does retrieve and update f_pos which should take
f_pos_lock and a patchset for this was posted but it didn't go anywhere.
It should probably hold that lock. See Jann's comments in the other
thread how that currently can lead to issues.
For getdents() not protecting f_pos is equally bad or worse. The patch
doesn't hold f_pos_lock and just updates f_pos prior _and_ post
iterate_dir() arguing that this race is fine. But again, f_version and
f_pos are consistent after each system call invocation.
But without that you can have a concurrent seek running and can end up
with an inconsistent f_pos position within the same system call. IOW,
you're breaking f_pos being in a well-known state. And you're not doing
that just for io_uring you're doing it for the regular system call
interface as well as both can be used on the same fd simultaneously.
So that's a no go imho.
> I don't see how different that is from a user spawning two threads and
> calling getdents64 + lseek or two getdents64 in parallel?
> (or any two other users of iterate_dir)
The difference is that in both cases f_pos_lock for both getdents and
lseek is held. So f_pos is in a good known state. You're not taking any
locks so now we're risking inconsistency within the same system call if
getdents and lseek run concurrently. Jens also mentioned that you could
even have this problem from within io_uring itself.
So tl;dr, there's no good reason to declare this an acceptable race
afaict. So either this is fixed properly or we're not doing it as far as
I'm concerned.
[1] https://lore.kernel.org/all/[email protected]/T/#m517583f23502f32b040c819d930359313b3db00c
[2] https://lore.kernel.org/io-uring/[email protected]
[3] https://lore.kernel.org/io-uring/[email protected]
Christian Brauner wrote on Wed, May 24, 2023 at 03:52:45PM +0200:
> The main objection in [1] was to allow specifying an arbitrary offset
> from userspace. What [3] did was to implement a pread() variant for
> directories, i.e., pgetdents(). That can't work in principle/is
> prohibitively complex. Which is what your series avoids by not allowing
> any offsets to be specified.
Yes.
> However, there's still a problem here. Updates to f_pos happen under an
> approriate lock to guarantee consistency of the position between calls
> that move the cursor position. In the normal read-write path io_uring
> doesn't concern itself with f_pos as it keeps local state in
> kiocb->ki_pos.
>
> But then it still does end up running into f_pos consistency problems
> for read-write because it does allow operating on the current f_pos if
> the offset if struct io_rw is set to -1.
>
> In that case it does retrieve and update f_pos which should take
> f_pos_lock and a patchset for this was posted but it didn't go anywhere.
> It should probably hold that lock. See Jann's comments in the other
> thread how that currently can lead to issues.
Assuming that is this mail:
https://lore.kernel.org/io-uring/CAG48ez1O9VxSuWuLXBjke23YxUA8EhMP+6RCHo5PNQBf3B0pDQ@mail.gmail.com/
So, ok, I didn't realize fdget_pos() actually acted as a lock on the
file's f_pos_lock (apparently only if FMODE_ATMOIC_POS is set? but it
looks set on most if not all directories through finish_open(), that
looks called consistently enough)
What was confusing is that default_llseek updates f_pos under the
inode_lock (write), and getdents also takes that lock (for read only in
shared implem), so I assumed getdents also was just protected by this
read lock, but I guess that was a bad assumption (as I kept pointing
out, a shared read lock isn't good enough, we definitely agree there)
In practice, in the non-registered file case io_uring is also calling
fdget, so the lock is held exactly the same as the syscall and I wasn't
so far off -- but we need to figure something for the registered file
case.
openat variants don't allow working with the registered variant at all
on the parent fd, so as far as I'm concerned I'd be happy setting the
same limitation for getdents: just say it acts on fd and not files, and
call it a day...
It'd also be possible to check if REQ_F_FIXED_FILE flag was set and
manually take the lock somehow but we don't have any primitive that
takes f_pos_lock from a file (the only place that takes it is fdget
which requires a fd), so I'd rather not add such a new exception.
I assume the other patch you mentioned about adding that lock was this
one:
https://lore.kernel.org/all/[email protected]/T/#m3609dc8057d0bc8e41ceab643e4d630f7b91bde6
and it just atkes the lock, but __fdget_pos also checks for
FMODE_ATOMIC_OPS and file_count and I'm not sure I understand how it
sets (f.flags & FDPUT_POS_UNLOCK) (for fdput_pos) so I'd rather not add
such a code path at this point..
So, ok, what do you think about just forbidding registered files?
I can't see where that wouldn't suffice but I might be missing something
else.
> For getdents() not protecting f_pos is equally bad or worse. The patch
> doesn't hold f_pos_lock and just updates f_pos prior _and_ post
> iterate_dir() arguing that this race is fine. But again, f_version and
> f_pos are consistent after each system call invocation.
>
> But without that you can have a concurrent seek running and can end up
> with an inconsistent f_pos position within the same system call. IOW,
> you're breaking f_pos being in a well-known state. And you're not doing
> that just for io_uring you're doing it for the regular system call
> interface as well as both can be used on the same fd simultaneously.
> So that's a no go imho.
(so seek is fine, but I agree two concurrent getdents on registered
files won't have the required lock)
--
Dominique Martinet | Asmadeus
On Thu, May 25, 2023 at 06:36:17AM +0900, Dominique Martinet wrote:
> Christian Brauner wrote on Wed, May 24, 2023 at 03:52:45PM +0200:
> > The main objection in [1] was to allow specifying an arbitrary offset
> > from userspace. What [3] did was to implement a pread() variant for
> > directories, i.e., pgetdents(). That can't work in principle/is
> > prohibitively complex. Which is what your series avoids by not allowing
> > any offsets to be specified.
>
> Yes.
>
> > However, there's still a problem here. Updates to f_pos happen under an
> > approriate lock to guarantee consistency of the position between calls
> > that move the cursor position. In the normal read-write path io_uring
> > doesn't concern itself with f_pos as it keeps local state in
> > kiocb->ki_pos.
> >
> > But then it still does end up running into f_pos consistency problems
> > for read-write because it does allow operating on the current f_pos if
> > the offset if struct io_rw is set to -1.
> >
> > In that case it does retrieve and update f_pos which should take
> > f_pos_lock and a patchset for this was posted but it didn't go anywhere.
> > It should probably hold that lock. See Jann's comments in the other
> > thread how that currently can lead to issues.
>
> Assuming that is this mail:
> https://lore.kernel.org/io-uring/CAG48ez1O9VxSuWuLXBjke23YxUA8EhMP+6RCHo5PNQBf3B0pDQ@mail.gmail.com/
>
> So, ok, I didn't realize fdget_pos() actually acted as a lock on the
> file's f_pos_lock (apparently only if FMODE_ATMOIC_POS is set? but it
> looks set on most if not all directories through finish_open(), that
> looks called consistently enough)
It's set for any regular file and directory.
>
> What was confusing is that default_llseek updates f_pos under the
> inode_lock (write), and getdents also takes that lock (for read only in
> shared implem), so I assumed getdents also was just protected by this
> read lock, but I guess that was a bad assumption (as I kept pointing
> out, a shared read lock isn't good enough, we definitely agree there)
>
>
> In practice, in the non-registered file case io_uring is also calling
> fdget, so the lock is held exactly the same as the syscall and I wasn't
No, it really isn't. fdget() doesn't take f_pos_lock at all:
fdget()
-> __fdget()
-> __fget_light()
-> __fget()
-> __fget_files()
-> __fget_files_rcu()
If that were true then any system call that passes an fd and uses
fdget() would try to acquire a mutex on f_pos_lock. We'd be serializing
every *at based system call on f_pos_lock whenever we have multiple fds
referring to the same file trying to operate on it concurrently.
We do have fdget_pos() and fdput_pos() as a special purpose fdget() for
a select group of system calls that require this synchronization.
> so far off -- but we need to figure something for the registered file
> case.
> openat variants don't allow working with the registered variant at all
> on the parent fd, so as far as I'm concerned I'd be happy setting the
> same limitation for getdents: just say it acts on fd and not files, and
> call it a day...
I don't follow. Also this is hacky so no.
The reason why io_uring *at implementations don't work with fixed files
is that the VFS interface expect regular fds. You could very well make
this work for fixed files but why. It would mean exposing a whole new
set of vfs helpers to io_uring and would probably involve nasty corner
cases.
Also the connection between regular and fixed files in io_uring is
pretty much fluent. While fixed files can only remain pinned in an
io_uring instance it requires that the caller explicitly gave up all
references in their fdtable to that struct file by closing all fds
referring to the same file.
But there's no guarantee. For example, if another thread dups the fd or
the caller sends the fd via SCM_RIGHTS to another process or the caller
simply doesn't close the fd or another thread gets an fd to the same
file from that task via pidfd_getfd before it closed it this doesn't hold.
So it's very well possible to have an fd and a fixed io_uring reference
referring to the same file. The first one can be used with the regular
system call interface and io_uring *at requests that forbid fixed files.
And the other one can be used for i_uring fixed file operations. Doesn't
matter if that shouldn't be done, it's possible afaict.
For regular and fixed files you also have the same problem from within
the same io_uring instance where you can have concurrent getdent
requests. You'd end up producing the exact same inconsistencies.
> It'd also be possible to check if REQ_F_FIXED_FILE flag was set and
> manually take the lock somehow but we don't have any primitive that
> takes f_pos_lock from a file (the only place that takes it is fdget
> which requires a fd), so I'd rather not add such a new exception.
> I assume the other patch you mentioned about adding that lock was this
> one:
> https://lore.kernel.org/all/[email protected]/T/#m3609dc8057d0bc8e41ceab643e4d630f7b91bde6
> and it just atkes the lock, but __fdget_pos also checks for
> FMODE_ATOMIC_OPS and file_count and I'm not sure I understand how it
> sets (f.flags & FDPUT_POS_UNLOCK) (for fdput_pos) so I'd rather not add
> such a code path at this point..
>
>
> So, ok, what do you think about just forbidding registered files?
> I can't see where that wouldn't suffice but I might be missing something
> else.
It doesn't help.
Christian Brauner wrote on Thu, May 25, 2023 at 11:22:08AM +0200:
> > What was confusing is that default_llseek updates f_pos under the
> > inode_lock (write), and getdents also takes that lock (for read only in
> > shared implem), so I assumed getdents also was just protected by this
> > read lock, but I guess that was a bad assumption (as I kept pointing
> > out, a shared read lock isn't good enough, we definitely agree there)
> >
> >
> > In practice, in the non-registered file case io_uring is also calling
> > fdget, so the lock is held exactly the same as the syscall and I wasn't
>
> No, it really isn't. fdget() doesn't take f_pos_lock at all:
>
> fdget()
> -> __fdget()
> -> __fget_light()
> -> __fget()
> -> __fget_files()
> -> __fget_files_rcu()
Ugh, I managed to not notice that I was looking at fdget_pos and that
it's not the same as fdget by the time I wrote two paragraphs... These
functions all have too many wrappers and too similar names for a quick
look before work.
> If that were true then any system call that passes an fd and uses
> fdget() would try to acquire a mutex on f_pos_lock. We'd be serializing
> every *at based system call on f_pos_lock whenever we have multiple fds
> referring to the same file trying to operate on it concurrently.
>
> We do have fdget_pos() and fdput_pos() as a special purpose fdget() for
> a select group of system calls that require this synchronization.
Right, that makes sense, and invalidates everything I said after that
anyway but it's not like looking stupid ever killed anyone.
Ok so it would require adding a new wrapper from struct file to struct
fd that'd eventually take the lock and set FDPUT_POS_UNLOCK for... not
fdput_pos but another function for that stopping short of fdput...
Then just call that around both vfs_llseek and vfs_getdents calls; which
is the easy part.
(Or possibly call mutex_lock directly like Dylan did in [1]...)
[1] https://lore.kernel.org/all/[email protected]/T/#m3609dc8057d0bc8e41ceab643e4d630f7b91bde6
I'll be honest though I'm thankful for your explanations but I think
I'll just do like Stefan and stop trying for now: the only reason I've
started this was because I wanted to play with io_uring for a new toy
project and it felt awkward without a getdents for crawling a tree; and
I'm long past the point where I should have thrown the towel and just
make that a sequential walk.
There's too many "conditional patches" (NOWAIT, end of dir indicator)
that I don't care about and require additional work to rebase
continuously so I'll just leave it up to someone else who does care.
So to that someone: feel free to continue from these branches (I've
included the fix for kernfs_fop_readdir that Dan Carpenter reported):
https://github.com/martinetd/linux/commits/io_uring_getdents
https://github.com/martinetd/liburing/commits/getdents
Or just start over, there's not that much code now hopefully the
baseline requirements have gotten a little bit clearer.
Sorry for stirring the mess and leaving halfway, if nobody does continue
I might send a v3 when I have more time/energy in a few months, but it
won't be quick.
--
Dominique
On Thu, May 25, 2023 at 08:00:02PM +0900, Dominique Martinet wrote:
> Christian Brauner wrote on Thu, May 25, 2023 at 11:22:08AM +0200:
> > > What was confusing is that default_llseek updates f_pos under the
> > > inode_lock (write), and getdents also takes that lock (for read only in
> > > shared implem), so I assumed getdents also was just protected by this
> > > read lock, but I guess that was a bad assumption (as I kept pointing
> > > out, a shared read lock isn't good enough, we definitely agree there)
> > >
> > >
> > > In practice, in the non-registered file case io_uring is also calling
> > > fdget, so the lock is held exactly the same as the syscall and I wasn't
> >
> > No, it really isn't. fdget() doesn't take f_pos_lock at all:
> >
> > fdget()
> > -> __fdget()
> > -> __fget_light()
> > -> __fget()
> > -> __fget_files()
> > -> __fget_files_rcu()
>
> Ugh, I managed to not notice that I was looking at fdget_pos and that
> it's not the same as fdget by the time I wrote two paragraphs... These
> functions all have too many wrappers and too similar names for a quick
> look before work.
>
> > If that were true then any system call that passes an fd and uses
> > fdget() would try to acquire a mutex on f_pos_lock. We'd be serializing
> > every *at based system call on f_pos_lock whenever we have multiple fds
> > referring to the same file trying to operate on it concurrently.
> >
> > We do have fdget_pos() and fdput_pos() as a special purpose fdget() for
> > a select group of system calls that require this synchronization.
>
> Right, that makes sense, and invalidates everything I said after that
> anyway but it's not like looking stupid ever killed anyone.
I strongly disagree with the looking stupid part. These callchains are
quite unwieldy and it's easy to get confused. Usually if you receive a
long mail about the semantics involved - as in the earlier thread - it
means there's landmines all over.
>
> Ok so it would require adding a new wrapper from struct file to struct
> fd that'd eventually take the lock and set FDPUT_POS_UNLOCK for... not
> fdput_pos but another function for that stopping short of fdput...
> Then just call that around both vfs_llseek and vfs_getdents calls; which
> is the easy part.
>
> (Or possibly call mutex_lock directly like Dylan did in [1]...)
> [1] https://lore.kernel.org/all/[email protected]/T/#m3609dc8057d0bc8e41ceab643e4d630f7b91bde6
We'd need a consistent story whatever it ends up being.
> I'll be honest though I'm thankful for your explanations but I think
> I'll just do like Stefan and stop trying for now: the only reason I've
> started this was because I wanted to play with io_uring for a new toy
> project and it felt awkward without a getdents for crawling a tree; and
> I'm long past the point where I should have thrown the towel and just
> make that a sequential walk.
> There's too many "conditional patches" (NOWAIT, end of dir indicator)
> that I don't care about and require additional work to rebase
> continuously so I'll just leave it up to someone else who does care.
>
> So to that someone: feel free to continue from these branches (I've
> included the fix for kernfs_fop_readdir that Dan Carpenter reported):
> https://github.com/martinetd/linux/commits/io_uring_getdents
> https://github.com/martinetd/liburing/commits/getdents
>
> Or just start over, there's not that much code now hopefully the
> baseline requirements have gotten a little bit clearer.
>
>
> Sorry for stirring the mess and leaving halfway, if nobody does continue
> I might send a v3 when I have more time/energy in a few months, but it
> won't be quick.
It's fine.
On 5/25/23 19:00, Dominique Martinet wrote:
> Christian Brauner wrote on Thu, May 25, 2023 at 11:22:08AM +0200:
>>> What was confusing is that default_llseek updates f_pos under the
>>> inode_lock (write), and getdents also takes that lock (for read only in
>>> shared implem), so I assumed getdents also was just protected by this
>>> read lock, but I guess that was a bad assumption (as I kept pointing
>>> out, a shared read lock isn't good enough, we definitely agree there)
>>>
>>>
>>> In practice, in the non-registered file case io_uring is also calling
>>> fdget, so the lock is held exactly the same as the syscall and I wasn't
>>
>> No, it really isn't. fdget() doesn't take f_pos_lock at all:
>>
>> fdget()
>> -> __fdget()
>> -> __fget_light()
>> -> __fget()
>> -> __fget_files()
>> -> __fget_files_rcu()
>
> Ugh, I managed to not notice that I was looking at fdget_pos and that
> it's not the same as fdget by the time I wrote two paragraphs... These
> functions all have too many wrappers and too similar names for a quick
> look before work.
>
>> If that were true then any system call that passes an fd and uses
>> fdget() would try to acquire a mutex on f_pos_lock. We'd be serializing
>> every *at based system call on f_pos_lock whenever we have multiple fds
>> referring to the same file trying to operate on it concurrently.
>>
>> We do have fdget_pos() and fdput_pos() as a special purpose fdget() for
>> a select group of system calls that require this synchronization.
>
> Right, that makes sense, and invalidates everything I said after that
> anyway but it's not like looking stupid ever killed anyone.
>
> Ok so it would require adding a new wrapper from struct file to struct
> fd that'd eventually take the lock and set FDPUT_POS_UNLOCK for... not
> fdput_pos but another function for that stopping short of fdput...
> Then just call that around both vfs_llseek and vfs_getdents calls; which
> is the easy part.
>
> (Or possibly call mutex_lock directly like Dylan did in [1]...)
> [1] https://lore.kernel.org/all/[email protected]/T/#m3609dc8057d0bc8e41ceab643e4d630f7b91bde6
>
>
>
> I'll be honest though I'm thankful for your explanations but I think
> I'll just do like Stefan and stop trying for now: the only reason I've
> started this was because I wanted to play with io_uring for a new toy
> project and it felt awkward without a getdents for crawling a tree; and
> I'm long past the point where I should have thrown the towel and just
> make that a sequential walk.
> There's too many "conditional patches" (NOWAIT, end of dir indicator)
> that I don't care about and require additional work to rebase
> continuously so I'll just leave it up to someone else who does care.
>
> So to that someone: feel free to continue from these branches (I've
> included the fix for kernfs_fop_readdir that Dan Carpenter reported):
> https://github.com/martinetd/linux/commits/io_uring_getdents
> https://github.com/martinetd/liburing/commits/getdents
>
> Or just start over, there's not that much code now hopefully the
> baseline requirements have gotten a little bit clearer.
>
>
> Sorry for stirring the mess and leaving halfway, if nobody does continue
> I might send a v3 when I have more time/energy in a few months, but it
> won't be quick.
>
Hi Dominique,
I'd like to take this if you don't mind.
Regards,
Hao
Hao Xu wrote on Tue, Jul 11, 2023 at 04:17:11PM +0800:
> > So to that someone: feel free to continue from these branches (I've
> > included the fix for kernfs_fop_readdir that Dan Carpenter reported):
> > https://github.com/martinetd/linux/commits/io_uring_getdents
> > https://github.com/martinetd/liburing/commits/getdents
> >
> > Or just start over, there's not that much code now hopefully the
> > baseline requirements have gotten a little bit clearer.
> >
> >
> > Sorry for stirring the mess and leaving halfway, if nobody does continue
> > I might send a v3 when I have more time/energy in a few months, but it
> > won't be quick.
>
> I'd like to take this if you don't mind.
Sure, I haven't been working on this lately, feel free to work on this.
Will be happy to review anything you send based on what came out of the
previous discussions to save Christian and others some time so you can
keep me in Cc if you'd like.
--
Dominique Martinet | Asmadeus