2021-11-18 23:57:56

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

Hi,

FAN_FS_ERROR was merged into Linus tree, and the PIDFD testcases reached
LTP. Therefore, I'm sending a new version of the FAN_FS_ERROR LTP
tests. This is the v4 of this patchset, and it applies the feedback of
the previous version.

Thanks,

---

Original cover letter:

FAN_FS_ERROR is a new (still unmerged) fanotify event to monitor
fileystem errors. This patchset introduces a new LTP test for this
feature.

Testing file system errors is slightly tricky, in particular because
they are mostly file system dependent. Since there are only patches for
ext4, I choose to make the test around it, since there wouldn't be much
to do with other file systems. The second challenge is how we cause the
file system errors, since there is no error injection for ext4 in Linux.
In this series, this is done by corrupting specific data in the
test device with the help of debugfs.

The FAN_FS_ERROR feature is flying around linux-ext4 and fsdevel, and
the latest version is available on the branch below:

https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-v9

A proper manpage description is also available on the respective mailing
list, or in the branch below:

https://gitlab.collabora.com/krisman/man-pages.git -b fan-fs-error

Please, let me know your thoughts.

Gabriel Krisman Bertazi (9):
syscalls: fanotify: Add macro to require specific mark types
syscalls: fanotify: Add macro to require specific events
syscalls/fanotify22: Introduce FAN_FS_ERROR test
syscalls/fanotify22: Validate the generic error info
syscalls/fanotify22: Validate incoming FID in FAN_FS_ERROR
syscalls/fanotify22: Support submission of debugfs commands
syscalls/fanotify22: Create a corrupted file
syscalls/fanotify22: Test file event with broken inode
syscalls/fanotify22: Test capture of multiple errors

configure.ac | 3 +-
testcases/kernel/syscalls/fanotify/.gitignore | 1 +
testcases/kernel/syscalls/fanotify/fanotify.h | 65 +++-
.../kernel/syscalls/fanotify/fanotify03.c | 4 +-
.../kernel/syscalls/fanotify/fanotify10.c | 3 +-
.../kernel/syscalls/fanotify/fanotify12.c | 3 +-
.../kernel/syscalls/fanotify/fanotify22.c | 314 ++++++++++++++++++
7 files changed, 385 insertions(+), 8 deletions(-)
create mode 100644 testcases/kernel/syscalls/fanotify/fanotify22.c

--
2.33.0



2021-11-18 23:58:08

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: [PATCH v4 3/9] syscalls/fanotify22: Introduce FAN_FS_ERROR test

fanotify22 is a new test validating the FAN_FS_ERROR file system error
event. This adds some basic structure for the next patches and a single
test of error reporting during filesystem abort

The strategy for error reporting testing in fanotify22 goes like this:

- Generate a broken filesystem
- Start FAN_FS_ERROR monitoring group
- Make the file system notice the error through ordinary operations
- Observe the event generated

FAN_FS_ERROR was added in the kernel by Linux commit
9709bd548f11 ("fanotify: Allow users to request FAN_FS_ERROR events").

Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Gabriel Krisman Bertazi <[email protected]>

---
Changes since v3:
- fanotify21 -> fanotify22 (Matthew)
Changes since v1:
- Move defines to header file.
- Move fanotify_mark(2) to do_test (Amir)
- Merge abort test here
---
testcases/kernel/syscalls/fanotify/.gitignore | 1 +
testcases/kernel/syscalls/fanotify/fanotify.h | 3 +
.../kernel/syscalls/fanotify/fanotify22.c | 140 ++++++++++++++++++
3 files changed, 144 insertions(+)
create mode 100644 testcases/kernel/syscalls/fanotify/fanotify22.c

diff --git a/testcases/kernel/syscalls/fanotify/.gitignore b/testcases/kernel/syscalls/fanotify/.gitignore
index 35e73b91e392..6d4ab4ca3c06 100644
--- a/testcases/kernel/syscalls/fanotify/.gitignore
+++ b/testcases/kernel/syscalls/fanotify/.gitignore
@@ -19,4 +19,5 @@
/fanotify19
/fanotify20
/fanotify21
+/fanotify22
/fanotify_child
diff --git a/testcases/kernel/syscalls/fanotify/fanotify.h b/testcases/kernel/syscalls/fanotify/fanotify.h
index 242245826004..3a8f950950e0 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify.h
+++ b/testcases/kernel/syscalls/fanotify/fanotify.h
@@ -127,6 +127,9 @@ static inline int safe_fanotify_mark(const char *file, const int lineno,
#ifndef FAN_OPEN_EXEC_PERM
#define FAN_OPEN_EXEC_PERM 0x00040000
#endif
+#ifndef FAN_FS_ERROR
+#define FAN_FS_ERROR 0x00008000
+#endif

/* Additional error status codes that can be returned to userspace */
#ifndef FAN_NOPIDFD
diff --git a/testcases/kernel/syscalls/fanotify/fanotify22.c b/testcases/kernel/syscalls/fanotify/fanotify22.c
new file mode 100644
index 000000000000..55e695b133d6
--- /dev/null
+++ b/testcases/kernel/syscalls/fanotify/fanotify22.c
@@ -0,0 +1,140 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2021 Collabora Ltd.
+ *
+ * Author: Gabriel Krisman Bertazi <[email protected]>
+ * Based on previous work by Amir Goldstein <[email protected]>
+ */
+
+/*\
+ * [Description]
+ * Check fanotify FAN_ERROR_FS events triggered by intentionally
+ * corrupted filesystems:
+ *
+ * - Generate a broken filesystem
+ * - Start FAN_FS_ERROR monitoring group
+ * - Make the file system notice the error through ordinary operations
+ * - Observe the event generated
+ */
+
+#define _GNU_SOURCE
+#include "config.h"
+
+#include <stdio.h>
+#include <sys/types.h>
+#include <errno.h>
+#include <string.h>
+#include <sys/mount.h>
+#include <sys/syscall.h>
+#include "tst_test.h"
+#include <sys/fanotify.h>
+#include <sys/types.h>
+#include <fcntl.h>
+
+#ifdef HAVE_SYS_FANOTIFY_H
+#include "fanotify.h"
+
+#define BUF_SIZE 256
+static char event_buf[BUF_SIZE];
+int fd_notify;
+
+#define MOUNT_PATH "test_mnt"
+
+static void trigger_fs_abort(void)
+{
+ SAFE_MOUNT(tst_device->dev, MOUNT_PATH, tst_device->fs_type,
+ MS_REMOUNT|MS_RDONLY, "abort");
+}
+
+static struct test_case {
+ char *name;
+ void (*trigger_error)(void);
+} testcases[] = {
+ {
+ .name = "Trigger abort",
+ .trigger_error = &trigger_fs_abort,
+ },
+};
+
+int check_error_event_metadata(struct fanotify_event_metadata *event)
+{
+ int fail = 0;
+
+ if (event->mask != FAN_FS_ERROR) {
+ fail++;
+ tst_res(TFAIL, "got unexpected event %llx",
+ (unsigned long long)event->mask);
+ }
+
+ if (event->fd != FAN_NOFD) {
+ fail++;
+ tst_res(TFAIL, "Weird FAN_FD %llx",
+ (unsigned long long)event->mask);
+ }
+ return fail;
+}
+
+void check_event(char *buf, size_t len, const struct test_case *ex)
+{
+ struct fanotify_event_metadata *event =
+ (struct fanotify_event_metadata *) buf;
+
+ if (len < FAN_EVENT_METADATA_LEN) {
+ tst_res(TFAIL, "No event metadata found");
+ return;
+ }
+
+ if (check_error_event_metadata(event))
+ return;
+
+ tst_res(TPASS, "Successfully received: %s", ex->name);
+}
+
+static void do_test(unsigned int i)
+{
+ const struct test_case *tcase = &testcases[i];
+ size_t read_len;
+
+ SAFE_FANOTIFY_MARK(fd_notify, FAN_MARK_ADD|FAN_MARK_FILESYSTEM,
+ FAN_FS_ERROR, AT_FDCWD, MOUNT_PATH);
+
+ tcase->trigger_error();
+
+ read_len = SAFE_READ(0, fd_notify, event_buf, BUF_SIZE);
+
+ SAFE_FANOTIFY_MARK(fd_notify, FAN_MARK_REMOVE|FAN_MARK_FILESYSTEM,
+ FAN_FS_ERROR, AT_FDCWD, MOUNT_PATH);
+
+ check_event(event_buf, read_len, tcase);
+}
+
+static void setup(void)
+{
+ REQUIRE_FANOTIFY_EVENTS_SUPPORTED_ON_FS(FAN_CLASS_NOTIF|FAN_REPORT_FID,
+ FAN_MARK_FILESYSTEM,
+ FAN_FS_ERROR, ".");
+
+ fd_notify = SAFE_FANOTIFY_INIT(FAN_CLASS_NOTIF|FAN_REPORT_FID,
+ O_RDONLY);
+}
+
+static void cleanup(void)
+{
+ if (fd_notify > 0)
+ SAFE_CLOSE(fd_notify);
+}
+
+static struct tst_test test = {
+ .test = do_test,
+ .tcnt = ARRAY_SIZE(testcases),
+ .setup = setup,
+ .cleanup = cleanup,
+ .mount_device = 1,
+ .mntpoint = MOUNT_PATH,
+ .needs_root = 1,
+ .dev_fs_type = "ext4"
+};
+
+#else
+ TST_TEST_TCONF("system doesn't have required fanotify support");
+#endif
--
2.33.0


2021-11-18 23:58:12

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: [PATCH v4 4/9] syscalls/fanotify22: Validate the generic error info

Implement some validation for the generic error info record emitted by
the kernel. The error number is fs-specific but, well, we only support
ext4 for now anyway.

Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Gabriel Krisman Bertazi <[email protected]>

---
Changes since v2:
- check for error record type in autotools (Amir)
Changes since v1:
- Move defines to header file.
---
configure.ac | 3 +-
testcases/kernel/syscalls/fanotify/fanotify.h | 32 ++++++++++++++++
.../kernel/syscalls/fanotify/fanotify22.c | 37 ++++++++++++++++++-
3 files changed, 70 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index 859aa9857021..a9dc2524966d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -160,7 +160,8 @@ AC_CHECK_MEMBERS([struct utsname.domainname],,,[
AC_CHECK_TYPES([enum kcmp_type],,,[#include <linux/kcmp.h>])
AC_CHECK_TYPES([struct acct_v3],,,[#include <sys/acct.h>])
AC_CHECK_TYPES([struct af_alg_iv, struct sockaddr_alg],,,[# include <linux/if_alg.h>])
-AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header],,,[#include <sys/fanotify.h>])
+AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header,
+ struct fanotify_event_info_error],[],[],[#include <sys/fanotify.h>])
AC_CHECK_TYPES([struct file_dedupe_range],,,[#include <linux/fs.h>])

AC_CHECK_TYPES([struct file_handle],,,[
diff --git a/testcases/kernel/syscalls/fanotify/fanotify.h b/testcases/kernel/syscalls/fanotify/fanotify.h
index 3a8f950950e0..6b0e68a23a6b 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify.h
+++ b/testcases/kernel/syscalls/fanotify/fanotify.h
@@ -181,6 +181,9 @@ typedef struct {
#ifndef FAN_EVENT_INFO_TYPE_PIDFD
#define FAN_EVENT_INFO_TYPE_PIDFD 4
#endif
+#ifndef FAN_EVENT_INFO_TYPE_ERROR
+#define FAN_EVENT_INFO_TYPE_ERROR 5
+#endif

#ifndef HAVE_STRUCT_FANOTIFY_EVENT_INFO_HEADER
struct fanotify_event_info_header {
@@ -205,6 +208,14 @@ struct fanotify_event_info_pidfd {
};
#endif /* HAVE_STRUCT_FANOTIFY_EVENT_INFO_PIDFD */

+#ifndef HAVE_STRUCT_FANOTIFY_EVENT_INFO_ERROR
+struct fanotify_event_info_error {
+ struct fanotify_event_info_header hdr;
+ __s32 error;
+ __u32 error_count;
+};
+#endif /* HAVE_STRUCT_FANOTIFY_EVENT_INFO_ERROR */
+
/* NOTE: only for struct fanotify_event_info_fid */
#ifdef HAVE_STRUCT_FANOTIFY_EVENT_INFO_FID_FSID___VAL
# define FSID_VAL_MEMBER(fsid, i) (fsid.__val[i])
@@ -418,4 +429,25 @@ static inline int fanotify_mark_supported_by_kernel(uint64_t flag)
fanotify_events_supported_by_kernel(mask, init_flags, mark_type)); \
} while (0)

+struct fanotify_event_info_header *get_event_info(
+ struct fanotify_event_metadata *event,
+ int info_type)
+{
+ struct fanotify_event_info_header *hdr = NULL;
+ char *start = (char *) event;
+ int off;
+
+ for (off = event->metadata_len; (off+sizeof(*hdr)) < event->event_len;
+ off += hdr->len) {
+ hdr = (struct fanotify_event_info_header *) &(start[off]);
+ if (hdr->info_type == info_type)
+ return hdr;
+ }
+ return NULL;
+}
+
+#define get_event_info_error(event) \
+ ((struct fanotify_event_info_error *) \
+ get_event_info((event), FAN_EVENT_INFO_TYPE_ERROR))
+
#endif /* __FANOTIFY_H__ */
diff --git a/testcases/kernel/syscalls/fanotify/fanotify22.c b/testcases/kernel/syscalls/fanotify/fanotify22.c
index 55e695b133d6..d324191c28c8 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify22.c
+++ b/testcases/kernel/syscalls/fanotify/fanotify22.c
@@ -48,14 +48,38 @@ static void trigger_fs_abort(void)

static struct test_case {
char *name;
+ int error;
+ unsigned int error_count;
void (*trigger_error)(void);
} testcases[] = {
{
.name = "Trigger abort",
.trigger_error = &trigger_fs_abort,
+ .error_count = 1,
+ .error = ESHUTDOWN,
},
};

+int check_error_event_info_error(struct fanotify_event_info_error *info_error,
+ const struct test_case *ex)
+{
+ int fail = 0;
+
+ if (info_error->error_count != ex->error_count) {
+ tst_res(TFAIL, "%s: Unexpected error_count (%d!=%d)",
+ ex->name, info_error->error_count, ex->error_count);
+ fail++;
+ }
+
+ if (info_error->error != ex->error) {
+ tst_res(TFAIL, "%s: Unexpected error code value (%d!=%d)",
+ ex->name, info_error->error, ex->error);
+ fail++;
+ }
+
+ return fail;
+}
+
int check_error_event_metadata(struct fanotify_event_metadata *event)
{
int fail = 0;
@@ -78,6 +102,8 @@ void check_event(char *buf, size_t len, const struct test_case *ex)
{
struct fanotify_event_metadata *event =
(struct fanotify_event_metadata *) buf;
+ struct fanotify_event_info_error *info_error;
+ int fail = 0;

if (len < FAN_EVENT_METADATA_LEN) {
tst_res(TFAIL, "No event metadata found");
@@ -87,7 +113,16 @@ void check_event(char *buf, size_t len, const struct test_case *ex)
if (check_error_event_metadata(event))
return;

- tst_res(TPASS, "Successfully received: %s", ex->name);
+ info_error = get_event_info_error(event);
+ if (info_error)
+ fail += check_error_event_info_error(info_error, ex);
+ else {
+ tst_res(TFAIL, "Generic error record not found");
+ fail++;
+ }
+
+ if (!fail)
+ tst_res(TPASS, "Successfully received: %s", ex->name);
}

static void do_test(unsigned int i)
--
2.33.0


2021-11-18 23:58:18

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: [PATCH v4 5/9] syscalls/fanotify22: Validate incoming FID in FAN_FS_ERROR

Verify the FID provided in the event. If the FH has size 0, this is
assumed to be a superblock error (i.e. null FH).

Reviewed-by: Petr Vorel <[email protected]>
Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Gabriel Krisman Bertazi <[email protected]>

---
Changes since v2:
- Move FILEID_INVALID define to header file (Amir)
- Don't use FSID_VAL_MEMBER (Petr)
Changes since v1:
- Move defines to header file.
- Use 0-len FH for sb error
---
testcases/kernel/syscalls/fanotify/fanotify.h | 8 +++
.../kernel/syscalls/fanotify/fanotify22.c | 60 +++++++++++++++++++
2 files changed, 68 insertions(+)

diff --git a/testcases/kernel/syscalls/fanotify/fanotify.h b/testcases/kernel/syscalls/fanotify/fanotify.h
index 6b0e68a23a6b..f7778bb36b4d 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify.h
+++ b/testcases/kernel/syscalls/fanotify/fanotify.h
@@ -255,6 +255,10 @@ static inline void fanotify_get_fid(const char *path, __kernel_fsid_t *fsid,
}
}

+#ifndef FILEID_INVALID
+#define FILEID_INVALID 0xff
+#endif
+
struct fanotify_fid_t {
__kernel_fsid_t fsid;
struct file_handle handle;
@@ -450,4 +454,8 @@ struct fanotify_event_info_header *get_event_info(
((struct fanotify_event_info_error *) \
get_event_info((event), FAN_EVENT_INFO_TYPE_ERROR))

+#define get_event_info_fid(event) \
+ ((struct fanotify_event_info_fid *) \
+ get_event_info((event), FAN_EVENT_INFO_TYPE_FID))
+
#endif /* __FANOTIFY_H__ */
diff --git a/testcases/kernel/syscalls/fanotify/fanotify22.c b/testcases/kernel/syscalls/fanotify/fanotify22.c
index d324191c28c8..eeee582f1268 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify22.c
+++ b/testcases/kernel/syscalls/fanotify/fanotify22.c
@@ -40,6 +40,9 @@ int fd_notify;

#define MOUNT_PATH "test_mnt"

+/* These expected FIDs are common to multiple tests */
+static struct fanotify_fid_t null_fid;
+
static void trigger_fs_abort(void)
{
SAFE_MOUNT(tst_device->dev, MOUNT_PATH, tst_device->fs_type,
@@ -50,6 +53,7 @@ static struct test_case {
char *name;
int error;
unsigned int error_count;
+ struct fanotify_fid_t *fid;
void (*trigger_error)(void);
} testcases[] = {
{
@@ -57,9 +61,43 @@ static struct test_case {
.trigger_error = &trigger_fs_abort,
.error_count = 1,
.error = ESHUTDOWN,
+ .fid = &null_fid,
},
};

+int check_error_event_info_fid(struct fanotify_event_info_fid *fid,
+ const struct test_case *ex)
+{
+ struct file_handle *fh = (struct file_handle *) &fid->handle;
+
+ if (memcmp(&fid->fsid, &ex->fid->fsid, sizeof(fid->fsid))) {
+ tst_res(TFAIL, "%s: Received bad FSID type (%x...!=%x...)",
+ ex->name, FSID_VAL_MEMBER(fid->fsid, 0),
+ ex->fid->fsid.val[0]);
+
+ return 1;
+ }
+ if (fh->handle_type != ex->fid->handle.handle_type) {
+ tst_res(TFAIL, "%s: Received bad file_handle type (%d!=%d)",
+ ex->name, fh->handle_type, ex->fid->handle.handle_type);
+ return 1;
+ }
+
+ if (fh->handle_bytes != ex->fid->handle.handle_bytes) {
+ tst_res(TFAIL, "%s: Received bad file_handle len (%d!=%d)",
+ ex->name, fh->handle_bytes, ex->fid->handle.handle_bytes);
+ return 1;
+ }
+
+ if (memcmp(fh->f_handle, ex->fid->handle.f_handle, fh->handle_bytes)) {
+ tst_res(TFAIL, "%s: Received wrong handle. "
+ "Expected (%x...) got (%x...) ", ex->name,
+ *(int *)ex->fid->handle.f_handle, *(int *)fh->f_handle);
+ return 1;
+ }
+ return 0;
+}
+
int check_error_event_info_error(struct fanotify_event_info_error *info_error,
const struct test_case *ex)
{
@@ -103,6 +141,7 @@ void check_event(char *buf, size_t len, const struct test_case *ex)
struct fanotify_event_metadata *event =
(struct fanotify_event_metadata *) buf;
struct fanotify_event_info_error *info_error;
+ struct fanotify_event_info_fid *info_fid;
int fail = 0;

if (len < FAN_EVENT_METADATA_LEN) {
@@ -121,6 +160,14 @@ void check_event(char *buf, size_t len, const struct test_case *ex)
fail++;
}

+ info_fid = get_event_info_fid(event);
+ if (info_fid)
+ fail += check_error_event_info_fid(info_fid, ex);
+ else {
+ tst_res(TFAIL, "FID record not found");
+ fail++;
+ }
+
if (!fail)
tst_res(TPASS, "Successfully received: %s", ex->name);
}
@@ -143,6 +190,17 @@ static void do_test(unsigned int i)
check_event(event_buf, read_len, tcase);
}

+static void init_null_fid(void)
+{
+ /* Use fanotify_save_fid to fill the fsid and overwrite the
+ * file_handler to create a null_fid
+ */
+ fanotify_save_fid(MOUNT_PATH, &null_fid);
+
+ null_fid.handle.handle_type = FILEID_INVALID;
+ null_fid.handle.handle_bytes = 0;
+}
+
static void setup(void)
{
REQUIRE_FANOTIFY_EVENTS_SUPPORTED_ON_FS(FAN_CLASS_NOTIF|FAN_REPORT_FID,
@@ -151,6 +209,8 @@ static void setup(void)

fd_notify = SAFE_FANOTIFY_INIT(FAN_CLASS_NOTIF|FAN_REPORT_FID,
O_RDONLY);
+
+ init_null_fid();
}

static void cleanup(void)
--
2.33.0


2021-11-18 23:58:19

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: [PATCH v4 6/9] syscalls/fanotify22: Support submission of debugfs commands

In order to test FAN_FS_ERROR, we want to corrupt the filesystem. The
easiest way to do it is by using debugfs. Add a small helper to issue
debugfs requests. Since most likely this will be the only testcase to
need this, don't bother making it a proper helper for now.

Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Gabriel Krisman Bertazi <[email protected]>
---
changes since v1:
- Add .needs_cmds to require debugfs
---
testcases/kernel/syscalls/fanotify/fanotify22.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/testcases/kernel/syscalls/fanotify/fanotify22.c b/testcases/kernel/syscalls/fanotify/fanotify22.c
index eeee582f1268..bb44ed55e96e 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify22.c
+++ b/testcases/kernel/syscalls/fanotify/fanotify22.c
@@ -49,6 +49,13 @@ static void trigger_fs_abort(void)
MS_REMOUNT|MS_RDONLY, "abort");
}

+static void do_debugfs_request(const char *dev, char *request)
+{
+ const char *cmd[] = {"debugfs", "-w", dev, "-R", request, NULL};
+
+ SAFE_CMD(cmd, NULL, NULL);
+}
+
static struct test_case {
char *name;
int error;
@@ -227,7 +234,11 @@ static struct tst_test test = {
.mount_device = 1,
.mntpoint = MOUNT_PATH,
.needs_root = 1,
- .dev_fs_type = "ext4"
+ .dev_fs_type = "ext4",
+ .needs_cmds = (const char *[]) {
+ "debugfs",
+ NULL
+ }
};

#else
--
2.33.0


2021-11-18 23:58:21

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: [PATCH v4 7/9] syscalls/fanotify22: Create a corrupted file

Allocate a test directory and corrupt it with debugfs. The corruption
is done by writing an invalid inode mode. This file can be later
looked up to trigger a corruption error.

Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Gabriel Krisman Bertazi <[email protected]>
---
testcases/kernel/syscalls/fanotify/fanotify22.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)

diff --git a/testcases/kernel/syscalls/fanotify/fanotify22.c b/testcases/kernel/syscalls/fanotify/fanotify22.c
index bb44ed55e96e..5d8b4eec74bd 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify22.c
+++ b/testcases/kernel/syscalls/fanotify/fanotify22.c
@@ -39,9 +39,12 @@ static char event_buf[BUF_SIZE];
int fd_notify;

#define MOUNT_PATH "test_mnt"
+#define BASE_DIR "internal_dir"
+#define BAD_DIR BASE_DIR"/bad_dir"

/* These expected FIDs are common to multiple tests */
static struct fanotify_fid_t null_fid;
+static struct fanotify_fid_t bad_file_fid;

static void trigger_fs_abort(void)
{
@@ -197,6 +200,18 @@ static void do_test(unsigned int i)
check_event(event_buf, read_len, tcase);
}

+static void pre_corrupt_fs(void)
+{
+ SAFE_MKDIR(MOUNT_PATH"/"BASE_DIR, 0777);
+ SAFE_MKDIR(MOUNT_PATH"/"BAD_DIR, 0777);
+
+ fanotify_save_fid(MOUNT_PATH"/"BAD_DIR, &bad_file_fid);
+
+ SAFE_UMOUNT(MOUNT_PATH);
+ do_debugfs_request(tst_device->dev, "sif " BAD_DIR " mode 0xff");
+ SAFE_MOUNT(tst_device->dev, MOUNT_PATH, tst_device->fs_type, 0, NULL);
+}
+
static void init_null_fid(void)
{
/* Use fanotify_save_fid to fill the fsid and overwrite the
@@ -213,6 +228,7 @@ static void setup(void)
REQUIRE_FANOTIFY_EVENTS_SUPPORTED_ON_FS(FAN_CLASS_NOTIF|FAN_REPORT_FID,
FAN_MARK_FILESYSTEM,
FAN_FS_ERROR, ".");
+ pre_corrupt_fs();

fd_notify = SAFE_FANOTIFY_INIT(FAN_CLASS_NOTIF|FAN_REPORT_FID,
O_RDONLY);
--
2.33.0


2021-11-18 23:58:25

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: [PATCH v4 8/9] syscalls/fanotify22: Test file event with broken inode

This test corrupts an inode entry with an invalid mode through debugfs
and then tries to access it. This should result in a ext4 error, which
we monitor through the fanotify group.

In order for this test to succeed, it requires a kernel fix introduced
by kernel commit 124e7c61deb2 ("ext4: fix error code saved on super
block during file system abort"), which is added to .tags.

Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Gabriel Krisman Bertazi <[email protected]>
---
.../kernel/syscalls/fanotify/fanotify22.c | 26 +++++++++++++++++++
1 file changed, 26 insertions(+)

diff --git a/testcases/kernel/syscalls/fanotify/fanotify22.c b/testcases/kernel/syscalls/fanotify/fanotify22.c
index 5d8b4eec74bd..9cd8c8f919b0 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify22.c
+++ b/testcases/kernel/syscalls/fanotify/fanotify22.c
@@ -34,6 +34,10 @@
#ifdef HAVE_SYS_FANOTIFY_H
#include "fanotify.h"

+#ifndef EFSCORRUPTED
+#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
+#endif
+
#define BUF_SIZE 256
static char event_buf[BUF_SIZE];
int fd_notify;
@@ -59,6 +63,17 @@ static void do_debugfs_request(const char *dev, char *request)
SAFE_CMD(cmd, NULL, NULL);
}

+static void tcase2_trigger_lookup(void)
+{
+ int ret;
+
+ /* SAFE_OPEN cannot be used here because we expect it to fail. */
+ ret = open(MOUNT_PATH"/"BAD_DIR, O_RDONLY, 0);
+ if (ret != -1 && errno != EUCLEAN)
+ tst_res(TFAIL, "Unexpected lookup result(%d) of %s (%d!=%d)",
+ ret, BAD_DIR, errno, EUCLEAN);
+}
+
static struct test_case {
char *name;
int error;
@@ -73,6 +88,13 @@ static struct test_case {
.error = ESHUTDOWN,
.fid = &null_fid,
},
+ {
+ .name = "Lookup of inode with invalid mode",
+ .trigger_error = &tcase2_trigger_lookup,
+ .error_count = 1,
+ .error = EFSCORRUPTED,
+ .fid = &bad_file_fid,
+ },
};

int check_error_event_info_fid(struct fanotify_event_info_fid *fid,
@@ -251,6 +273,10 @@ static struct tst_test test = {
.mntpoint = MOUNT_PATH,
.needs_root = 1,
.dev_fs_type = "ext4",
+ .tags = (const struct tst_tag[]) {
+ {"linux-git", "124e7c61deb2"},
+ {}
+ },
.needs_cmds = (const char *[]) {
"debugfs",
NULL
--
2.33.0


2021-11-18 23:58:31

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: [PATCH v4 9/9] syscalls/fanotify22: Test capture of multiple errors

When multiple FS errors occur, only the first is stored. This testcase
validates this behavior by issuing two different errors and making sure
only the first is stored, while the second is simply accumulated in
error_count.

Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Gabriel Krisman Bertazi <[email protected]>
---
.../kernel/syscalls/fanotify/fanotify22.c | 26 +++++++++++++++++++
1 file changed, 26 insertions(+)

diff --git a/testcases/kernel/syscalls/fanotify/fanotify22.c b/testcases/kernel/syscalls/fanotify/fanotify22.c
index 9cd8c8f919b0..b42e96e8263e 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify22.c
+++ b/testcases/kernel/syscalls/fanotify/fanotify22.c
@@ -74,6 +74,18 @@ static void tcase2_trigger_lookup(void)
ret, BAD_DIR, errno, EUCLEAN);
}

+static void tcase3_trigger(void)
+{
+ trigger_fs_abort();
+ tcase2_trigger_lookup();
+}
+
+static void tcase4_trigger(void)
+{
+ tcase2_trigger_lookup();
+ trigger_fs_abort();
+}
+
static struct test_case {
char *name;
int error;
@@ -95,6 +107,20 @@ static struct test_case {
.error = EFSCORRUPTED,
.fid = &bad_file_fid,
},
+ {
+ .name = "Multiple error submission",
+ .trigger_error = &tcase3_trigger,
+ .error_count = 2,
+ .error = ESHUTDOWN,
+ .fid = &null_fid,
+ },
+ {
+ .name = "Multiple error submission 2",
+ .trigger_error = &tcase4_trigger,
+ .error_count = 2,
+ .error = EFSCORRUPTED,
+ .fid = &bad_file_fid,
+ }
};

int check_error_event_info_fid(struct fanotify_event_info_fid *fid,
--
2.33.0


2021-11-19 05:38:57

by Amir Goldstein

[permalink] [raw]
Subject: Re: [PATCH v4 4/9] syscalls/fanotify22: Validate the generic error info

On Fri, Nov 19, 2021 at 1:58 AM Gabriel Krisman Bertazi
<[email protected]> wrote:
>
> Implement some validation for the generic error info record emitted by
> the kernel. The error number is fs-specific but, well, we only support
> ext4 for now anyway.
>
> Reviewed-by: Amir Goldstein <[email protected]>
> Signed-off-by: Gabriel Krisman Bertazi <[email protected]>
>
> ---
> Changes since v2:
> - check for error record type in autotools (Amir)
> Changes since v1:
> - Move defines to header file.
> ---
> configure.ac | 3 +-
> testcases/kernel/syscalls/fanotify/fanotify.h | 32 ++++++++++++++++
> .../kernel/syscalls/fanotify/fanotify22.c | 37 ++++++++++++++++++-
> 3 files changed, 70 insertions(+), 2 deletions(-)
>
> diff --git a/configure.ac b/configure.ac
> index 859aa9857021..a9dc2524966d 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -160,7 +160,8 @@ AC_CHECK_MEMBERS([struct utsname.domainname],,,[
> AC_CHECK_TYPES([enum kcmp_type],,,[#include <linux/kcmp.h>])
> AC_CHECK_TYPES([struct acct_v3],,,[#include <sys/acct.h>])
> AC_CHECK_TYPES([struct af_alg_iv, struct sockaddr_alg],,,[# include <linux/if_alg.h>])
> -AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header],,,[#include <sys/fanotify.h>])
> +AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header,
> + struct fanotify_event_info_error],[],[],[#include <sys/fanotify.h>])

Doh! it seems like fanotify_event_info_pidfd was dropped between v2 ->
v3 in Matthew's patches.

Petr,

Can you please fix this.

Thanks,
Amir.

2021-11-19 05:48:41

by Amir Goldstein

[permalink] [raw]
Subject: Re: [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

On Fri, Nov 19, 2021 at 1:57 AM Gabriel Krisman Bertazi
<[email protected]> wrote:
>
> Hi,
>
> FAN_FS_ERROR was merged into Linus tree, and the PIDFD testcases reached
> LTP. Therefore, I'm sending a new version of the FAN_FS_ERROR LTP
> tests. This is the v4 of this patchset, and it applies the feedback of
> the previous version.
>
> Thanks,
>
> ---
>
> Original cover letter:
>
> FAN_FS_ERROR is a new (still unmerged) fanotify event to monitor
> fileystem errors. This patchset introduces a new LTP test for this
> feature.
>
> Testing file system errors is slightly tricky, in particular because
> they are mostly file system dependent. Since there are only patches for
> ext4, I choose to make the test around it, since there wouldn't be much
> to do with other file systems. The second challenge is how we cause the
> file system errors, since there is no error injection for ext4 in Linux.
> In this series, this is done by corrupting specific data in the
> test device with the help of debugfs.
>
> The FAN_FS_ERROR feature is flying around linux-ext4 and fsdevel, and
> the latest version is available on the branch below:
>
> https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-v9
>
> A proper manpage description is also available on the respective mailing
> list, or in the branch below:
>
> https://gitlab.collabora.com/krisman/man-pages.git -b fan-fs-error
>
> Please, let me know your thoughts.
>

Gabriel,

Can you please push these v4 patches to your gitlab tree?

Thanks,
Amir.

2021-11-19 19:29:06

by Petr Vorel

[permalink] [raw]
Subject: Re: [PATCH v4 4/9] syscalls/fanotify22: Validate the generic error info

...
> > -AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header],,,[#include <sys/fanotify.h>])
> > +AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header,
> > + struct fanotify_event_info_error],[],[],[#include <sys/fanotify.h>])

> Doh! it seems like fanotify_event_info_pidfd was dropped between v2 ->
> v3 in Matthew's patches.

> Petr,

> Can you please fix this.

Hi Amir,

thanks for info, sure I'll fix it on Monday.

Kind regards,
Petr

> Thanks,
> Amir.

2021-11-19 19:29:20

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: Re: [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

Amir Goldstein <[email protected]> writes:

> On Fri, Nov 19, 2021 at 1:57 AM Gabriel Krisman Bertazi
> <[email protected]> wrote:
>>
>> Hi,
>>
>> FAN_FS_ERROR was merged into Linus tree, and the PIDFD testcases reached
>> LTP. Therefore, I'm sending a new version of the FAN_FS_ERROR LTP
>> tests. This is the v4 of this patchset, and it applies the feedback of
>> the previous version.
>>
>> Thanks,
>>
>> ---
>>
>> Original cover letter:
>>
>> FAN_FS_ERROR is a new (still unmerged) fanotify event to monitor
>> fileystem errors. This patchset introduces a new LTP test for this
>> feature.
>>
>> Testing file system errors is slightly tricky, in particular because
>> they are mostly file system dependent. Since there are only patches for
>> ext4, I choose to make the test around it, since there wouldn't be much
>> to do with other file systems. The second challenge is how we cause the
>> file system errors, since there is no error injection for ext4 in Linux.
>> In this series, this is done by corrupting specific data in the
>> test device with the help of debugfs.
>>
>> The FAN_FS_ERROR feature is flying around linux-ext4 and fsdevel, and
>> the latest version is available on the branch below:
>>
>> https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-v9
>>
>> A proper manpage description is also available on the respective mailing
>> list, or in the branch below:
>>
>> https://gitlab.collabora.com/krisman/man-pages.git -b fan-fs-error
>>
>> Please, let me know your thoughts.
>>
>
> Gabriel,
>
> Can you please push these v4 patches to your gitlab tree?

Hi Amir,

I have pushed v4 to :

https://gitlab.collabora.com/krisman/ltp.git -b fan-fs-error_v4

Thank you!

--
Gabriel Krisman Bertazi

2021-11-19 22:39:27

by Petr Vorel

[permalink] [raw]
Subject: Re: [PATCH v4 4/9] syscalls/fanotify22: Validate the generic error info

Hi Amir,

> ...
> > > -AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header],,,[#include <sys/fanotify.h>])
> > > +AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header,
> > > + struct fanotify_event_info_error],[],[],[#include <sys/fanotify.h>])

> > Doh! it seems like fanotify_event_info_pidfd was dropped between v2 ->
> > v3 in Matthew's patches.

> > Petr,

> > Can you please fix this.

> Hi Amir,

> thanks for info, sure I'll fix it on Monday.
Fixed now. Thanks!

Kind regards,
Petr

> Kind regards,
> Petr

> > Thanks,
> > Amir.

2021-11-20 10:44:11

by Amir Goldstein

[permalink] [raw]
Subject: Re: [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

> >> A proper manpage description is also available on the respective mailing
> >> list, or in the branch below:
> >>
> >> https://gitlab.collabora.com/krisman/man-pages.git -b fan-fs-error
> >>
> >> Please, let me know your thoughts.
> >>
> >
> > Gabriel,
> >
> > Can you please push these v4 patches to your gitlab tree?
>
> Hi Amir,
>
> I have pushed v4 to :
>
> https://gitlab.collabora.com/krisman/ltp.git -b fan-fs-error_v4
>

Thanks. I've based my fan_rename ltp branch on this.
I would like to do the same for the man-page update branch.
However, Matthew's man page updates for v5.15 are conflicting
with your changes, so after Matthew posts v2 of his man page patch,
please rebase your changes on top of his branch.

Ideally, you could have waited until Matthew's changes are merged
upstream, like you did for ltp before rebasing, but man-pages maintainers
are quite behind on merging updates, so we will need to collaborate
between us in the meanwhile.

Thanks,
Amir.

2021-11-22 07:47:50

by Petr Vorel

[permalink] [raw]
Subject: Re: [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

Hi all,

<snip>
> Hi Amir,

> I have pushed v4 to :

> https://gitlab.collabora.com/krisman/ltp.git -b fan-fs-error_v4

FYI I've rebased it on my fix 3b2ea2e00 ("configure.ac: Add struct
fanotify_event_info_pidfd check")

https://github.com/linux-test-project/ltp.git -b gertazi/fanotify21.v4.fixes

diff to krisman/fan-fs-error_v4:

diff --git configure.ac configure.ac
index a9dc25249..d25183368 100644
--- configure.ac
+++ configure.ac
@@ -160,8 +160,8 @@ AC_CHECK_MEMBERS([struct utsname.domainname],,,[
AC_CHECK_TYPES([enum kcmp_type],,,[#include <linux/kcmp.h>])
AC_CHECK_TYPES([struct acct_v3],,,[#include <sys/acct.h>])
AC_CHECK_TYPES([struct af_alg_iv, struct sockaddr_alg],,,[# include <linux/if_alg.h>])
-AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header,
- struct fanotify_event_info_error],[],[],[#include <sys/fanotify.h>])
+AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_error,
+ struct fanotify_event_info_header, struct fanotify_event_info_pidfd],,,[#include <sys/fanotify.h>])
AC_CHECK_TYPES([struct file_dedupe_range],,,[#include <linux/fs.h>])

AC_CHECK_TYPES([struct file_handle],,,[

Kind regards,
Petr

2021-11-22 17:35:10

by Gabriel Krisman Bertazi

[permalink] [raw]
Subject: Re: [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

Petr Vorel <[email protected]> writes:

> Hi all,
>
> <snip>
>> Hi Amir,
>
>> I have pushed v4 to :
>
>> https://gitlab.collabora.com/krisman/ltp.git -b fan-fs-error_v4
>
> FYI I've rebased it on my fix 3b2ea2e00 ("configure.ac: Add struct
> fanotify_event_info_pidfd check")
>
> https://github.com/linux-test-project/ltp.git -b gertazi/fanotify21.v4.fixes
>
> diff to krisman/fan-fs-error_v4:

Petr,

Should I send a v5 or is v4 getting picked up and merged with the fixup
hunk?

Thanks,

--
Gabriel Krisman Bertazi

2021-11-22 21:09:19

by Petr Vorel

[permalink] [raw]
Subject: Re: [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

Hi all,
> Petr Vorel <[email protected]> writes:

> > Hi all,

> > <snip>
> >> Hi Amir,

> >> I have pushed v4 to :

> >> https://gitlab.collabora.com/krisman/ltp.git -b fan-fs-error_v4

> > FYI I've rebased it on my fix 3b2ea2e00 ("configure.ac: Add struct
> > fanotify_event_info_pidfd check")

> > https://github.com/linux-test-project/ltp.git -b gertazi/fanotify21.v4.fixes

> > diff to krisman/fan-fs-error_v4:

> Petr,

> Should I send a v5 or is v4 getting picked up and merged with the fixup
> hunk?
No need to sent v4, I'll merge it from my branch. This is info for Amir, which
wanted to use your git tree to base his patchset on (if it wasn't relevant only
to patches for man-pages).

Kind regards,
Petr

2021-11-24 10:40:03

by Petr Vorel

[permalink] [raw]
Subject: Re: [LTP] [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

Hi all,

<snip>
> > Hi Amir,

> > I have pushed v4 to :

> > https://gitlab.collabora.com/krisman/ltp.git -b fan-fs-error_v4

> FYI I've rebased it on my fix 3b2ea2e00 ("configure.ac: Add struct
> fanotify_event_info_pidfd check")

> https://github.com/linux-test-project/ltp.git -b gertazi/fanotify21.v4.fixes

FYI I removed branch from official LTP repository and put it to my fork
https://github.com/pevik/ltp.git -b fan-fs-error_v4.fixes

Kind regards,
Petr

> diff to krisman/fan-fs-error_v4:

> diff --git configure.ac configure.ac
> index a9dc25249..d25183368 100644
> --- configure.ac
> +++ configure.ac
> @@ -160,8 +160,8 @@ AC_CHECK_MEMBERS([struct utsname.domainname],,,[
> AC_CHECK_TYPES([enum kcmp_type],,,[#include <linux/kcmp.h>])
> AC_CHECK_TYPES([struct acct_v3],,,[#include <sys/acct.h>])
> AC_CHECK_TYPES([struct af_alg_iv, struct sockaddr_alg],,,[# include <linux/if_alg.h>])
> -AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_header,
> - struct fanotify_event_info_error],[],[],[#include <sys/fanotify.h>])
> +AC_CHECK_TYPES([struct fanotify_event_info_fid, struct fanotify_event_info_error,
> + struct fanotify_event_info_header, struct fanotify_event_info_pidfd],,,[#include <sys/fanotify.h>])
> AC_CHECK_TYPES([struct file_dedupe_range],,,[#include <linux/fs.h>])

> AC_CHECK_TYPES([struct file_handle],,,[

> Kind regards,
> Petr

2021-12-20 11:49:08

by Amir Goldstein

[permalink] [raw]
Subject: Re: [LTP] [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

On Wed, Nov 24, 2021 at 12:40 PM Petr Vorel <[email protected]> wrote:
>
> Hi all,
>
> <snip>
> > > Hi Amir,
>
> > > I have pushed v4 to :
>
> > > https://gitlab.collabora.com/krisman/ltp.git -b fan-fs-error_v4
>
> > FYI I've rebased it on my fix 3b2ea2e00 ("configure.ac: Add struct
> > fanotify_event_info_pidfd check")
>
> > https://github.com/linux-test-project/ltp.git -b gertazi/fanotify21.v4.fixes
>
> FYI I removed branch from official LTP repository and put it to my fork
> https://github.com/pevik/ltp.git -b fan-fs-error_v4.fixes
>

Hi Petr,

Are you waiting with this merge for after release of v5.16?
or is it just waiting behind other work?

Just asking out of curiosity.
I've based my tests for fan_rename (queued for v5.17) on top of your branch.

Thanks,
Amir.

2021-12-20 18:07:39

by Petr Vorel

[permalink] [raw]
Subject: Re: [LTP] [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

Hi Amir,

[ Cc Cyril and Richie ]

> On Wed, Nov 24, 2021 at 12:40 PM Petr Vorel <[email protected]> wrote:

> > Hi all,

> > <snip>
> > > > Hi Amir,

> > > > I have pushed v4 to :

> > > > https://gitlab.collabora.com/krisman/ltp.git -b fan-fs-error_v4

> > > FYI I've rebased it on my fix 3b2ea2e00 ("configure.ac: Add struct
> > > fanotify_event_info_pidfd check")

> > > https://github.com/linux-test-project/ltp.git -b gertazi/fanotify21.v4.fixes

> > FYI I removed branch from official LTP repository and put it to my fork
> > https://github.com/pevik/ltp.git -b fan-fs-error_v4.fixes


> Hi Petr,

> Are you waiting with this merge for after release of v5.16?
> or is it just waiting behind other work?
Yes. Thanks for this input, we're just discussing our policy about tests for new
(kernel) release functionality. First we agreed to wait [1] (due problems
described in [2]), Richie is suggesting to merge earlier [2], although Cyril
had doubts it's worth of the work [3].

Kind regards,
Petr

> Just asking out of curiosity.
> I've based my tests for fan_rename (queued for v5.17) on top of your branch.

> Thanks,
> Amir.

[1] https://lore.kernel.org/ltp/[email protected]/
[2] https://lore.kernel.org/ltp/[email protected]/
[3] https://lore.kernel.org/ltp/[email protected]/

2022-01-10 15:17:00

by Petr Vorel

[permalink] [raw]
Subject: Re: [PATCH v4 0/9] Test the new fanotify FAN_FS_ERROR event

Hi all,

v5.16 released => patchset merged.
Thanks!

Kind regards,
Petr