Hi,
change v3->v4:
new commit "nfs05.sh: Lower down the default values"
to fix nfs05.sh failure on Btrfs.
I'd like to get this change into upcoming LTP release.
Kind regards,
Petr
Petr Vorel (4):
nfs_lib.sh: Cleanup local and remote directories setup
nfs_lib.sh: Unexport on proper side on netns
nfs05.sh: Lower down the default values
nfs: Run on all filesystems
testcases/network/nfs/nfs_stress/nfs05.sh | 4 +-
testcases/network/nfs/nfs_stress/nfs_lib.sh | 82 +++++++++++++++------
2 files changed, 63 insertions(+), 23 deletions(-)
--
2.40.0
f6b267055 changed exportfs run locally on netns, therefore unexporting
should be also run at the same namespace. This is not problematic now,
but will be a problem with TST_ALL_FILESYSTEMS=1, which is sensitive for
timing.
Fixes: f6b267055 ("nfs_lib.sh: run exportfs at "server side" in LTP_NETNS case")
Signed-off-by: Petr Vorel <[email protected]>
---
The same as in v3.
testcases/network/nfs/nfs_stress/nfs_lib.sh | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/testcases/network/nfs/nfs_stress/nfs_lib.sh b/testcases/network/nfs/nfs_stress/nfs_lib.sh
index 1b5604ab5..042fea5e4 100644
--- a/testcases/network/nfs/nfs_stress/nfs_lib.sh
+++ b/testcases/network/nfs/nfs_stress/nfs_lib.sh
@@ -215,8 +215,16 @@ nfs_cleanup()
for i in $VERSION; do
type=$(get_socket_type $n)
remote_dir="$(get_remote_dir $i $type)"
- tst_rhost_run -c "test -d $remote_dir && exportfs -u *:$remote_dir"
- tst_rhost_run -c "test -d $remote_dir && rm -rf $remote_dir"
+
+ if tst_net_use_netns; then
+ if test -d $remote_dir; then
+ exportfs -u *:$remote_dir
+ rm -rf $remote_dir
+ fi
+ else
+ tst_rhost_run -c "test -d $remote_dir && exportfs -u *:$remote_dir"
+ tst_rhost_run -c "test -d $remote_dir && rm -rf $remote_dir"
+ fi
n=$(( n + 1 ))
done
}
--
2.40.0
Logic for creating local and remote directories was on more places.
Create get_local_dir() and get_remote_dir() functions to keep it on
single place.
local dir is needed in nfs_mount(), but was defined in nfs_setup()
and reused local variable with shell inheritance (ugly!), because there
were all parameters from loop. Similarly, remote dir is needed in
both nfs_mount() and nfs_setup_server(), but created with shell
inheritance in nfs_setup(). Pass these params to nfs_mount() and
nfs_setup_server() and define variables with new functions
get_local_dir() and get_remote_dir().
Use get_remote_dir() in nfs_get_remote_path().
Move cd to local directory to the end of nfs_mount() (it used to cd
after nfs_mount(), but only if -v parameter contained single version,
but it does not harm to always cd).
Signed-off-by: Petr Vorel <[email protected]>
---
The same as in v3.
testcases/network/nfs/nfs_stress/nfs_lib.sh | 52 ++++++++++++++-------
1 file changed, 34 insertions(+), 18 deletions(-)
diff --git a/testcases/network/nfs/nfs_stress/nfs_lib.sh b/testcases/network/nfs/nfs_stress/nfs_lib.sh
index af7d46a21..1b5604ab5 100644
--- a/testcases/network/nfs/nfs_stress/nfs_lib.sh
+++ b/testcases/network/nfs/nfs_stress/nfs_lib.sh
@@ -1,6 +1,6 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
-# Copyright (c) Linux Test Project, 2016-2022
+# Copyright (c) Linux Test Project, 2016-2023
# Copyright (c) 2015-2018 Oracle and/or its affiliates. All Rights Reserved.
# Copyright (c) International Business Machines Corp., 2001
@@ -53,6 +53,24 @@ get_socket_type()
done
}
+# directory mounted by NFS client
+get_local_dir()
+{
+ local v="$1"
+ local n="$2"
+
+ echo "$TST_TMPDIR/$v/$n"
+}
+
+# directory on NFS server
+get_remote_dir()
+{
+ local v="$1"
+ local n="$2"
+
+ echo "$TST_TMPDIR/$v/$n"
+}
+
nfs_get_remote_path()
{
local v
@@ -63,7 +81,7 @@ nfs_get_remote_path()
done
v=${1:-$v}
- echo "$TST_TMPDIR/$v/$type"
+ echo "$(get_remote_dir $v $type)"
}
nfs_server_udp_enabled()
@@ -78,8 +96,8 @@ nfs_server_udp_enabled()
nfs_setup_server()
{
-
- local fsid="$1"
+ local remote_dir="$1"
+ local fsid="$2"
local export_cmd="exportfs -i -o fsid=$fsid,no_root_squash,rw *:$remote_dir"
[ -z "$fsid" ] && tst_brk TBROK "empty fsid"
@@ -97,10 +115,14 @@ nfs_setup_server()
nfs_mount()
{
- local opts="$1"
+ local local_dir="$1"
+ local remote_dir="$2"
+ local opts="$3"
local host_type=rhost
local mount_dir
+ mkdir -p "$local_dir"
+
tst_net_use_netns && host_type=
if [ $TST_IPV6 ]; then
@@ -131,6 +153,8 @@ nfs_mount()
tst_brk TBROK "mount command failed"
fi
+
+ cd "$local_dir"
}
nfs_setup()
@@ -162,20 +186,12 @@ nfs_setup()
tst_brk TCONF "UDP support disabled on NFS server"
fi
- local_dir="$TST_TMPDIR/$i/$n"
- remote_dir="$TST_TMPDIR/$i/$type"
- mkdir -p $local_dir
-
- nfs_setup_server $(($$ + n))
-
- nfs_mount "-o proto=$type,vers=$i"
+ remote_dir="$(get_remote_dir $i $type)"
+ nfs_setup_server "$remote_dir" "$(($$ + n))"
+ nfs_mount "$(get_local_dir $i $n)" "$remote_dir" "-o proto=$type,vers=$i"
n=$(( n + 1 ))
done
-
- if [ "$n" -eq 1 ]; then
- cd ${VERSION}/0
- fi
}
nfs_cleanup()
@@ -190,7 +206,7 @@ nfs_cleanup()
local n=0
for i in $VERSION; do
- local_dir="$TST_TMPDIR/$i/$n"
+ local_dir="$(get_local_dir $i $n)"
grep -q "$local_dir" /proc/mounts && umount $local_dir
n=$(( n + 1 ))
done
@@ -198,7 +214,7 @@ nfs_cleanup()
n=0
for i in $VERSION; do
type=$(get_socket_type $n)
- remote_dir="$TST_TMPDIR/$i/$type"
+ remote_dir="$(get_remote_dir $i $type)"
tst_rhost_run -c "test -d $remote_dir && exportfs -u *:$remote_dir"
tst_rhost_run -c "test -d $remote_dir && rm -rf $remote_dir"
n=$(( n + 1 ))
--
2.40.0
Use variables:
* TST_ALL_FILESYSTEMS=1 to run on all filesystems
* TST_FORMAT_DEVICE=1 to get loop device formatted
* TST_MOUNT_DEVICE=1 to get it mounted
Filesystems (tested the usual LTP way on loop device) are used for
server side (exportfs), client side (NFS mount) is kept outside of it.
For some reason umounting needs some time before NFS server stops using
underlying loop device. Also exportfs needs time before files can be
removed. Otherwise second umounting fails:
nfs07 4 TINFO: Cleaning up testcase
umount: /var/tmp/LTP_nfs07.FNZ7yCbqZe/mntpoint: target is busy.
nfs07 4 TINFO: umount(/var/tmp/LTP_nfs07.FNZ7yCbqZe/mntpoint) failed, try 1 ...
nfs07 4 TINFO: Likely gvfsd-trash is probing newly mounted fs, kill it to speed up tests.
umount: /var/tmp/LTP_nfs07.FNZ7yCbqZe/mntpoint: target is busy.
Solved with adding two sleeps for 1 sec (using less is not enough,
specially for nfs07.sh).
Skipping some problematic filesystems:
* exfat
Although it works on some systems (e.g. openSUSE Tumbleweed
with kernel 6.2.8-1-default, nfs-utils 2.6.2, exfatprogs 1.2.0), it
fails on other systems (e.g. SLES 15-SP4 with kernel 5.14.21, nfs-utils
2.1.1, exfatprogs 1.0.4 or Debian 12 bookworm with kernel 6.1.0-6-amd64,
nfs-utils 2.6.2, exfatprogs 1.2.0)
* tmpfs on nfs-utils < 2
tmpfs fails on nfs-utils 1.3.3:
nfs07 1 TINFO: mount.nfs: (linux nfs-utils 1.3.3)
nfs07 1 TINFO: setup NFSv4.2, socket type tcp
nfs07 1 TINFO: Mounting NFS: mount -v -t nfs -o proto=tcp,vers=4.2 10.0.0.2:/tmp/LTP_nfs07.cex71Q5bxw/mntpoint/4.2/tcp /tmp/LTP_nfs07.cex71Q5bxw/4.2/0
mount.nfs: mount(2): No such file or directory
mount.nfs: mounting 10.0.0.2:/tmp/LTP_nfs07.cex71Q5bxw/mntpoint/4.2/tcp failed, reason given by server: No such file or directory
mount.nfs: timeout set for Mon Apr 24 21:34:02 2023
mount.nfs: trying text-based options 'proto=tcp,vers=4.2,addr=10.0.0.2,clientaddr=10.0.0.1'
nfs07 1 TBROK: mount command failed
But it works on nfs-utils 2.1.1 on SLE15-SP4.
Signed-off-by: Petr Vorel <[email protected]>
---
The same as in v3.
testcases/network/nfs/nfs_stress/nfs_lib.sh | 20 ++++++++++++++++++--
1 file changed, 18 insertions(+), 2 deletions(-)
diff --git a/testcases/network/nfs/nfs_stress/nfs_lib.sh b/testcases/network/nfs/nfs_stress/nfs_lib.sh
index 042fea5e4..1c6657a14 100644
--- a/testcases/network/nfs/nfs_stress/nfs_lib.sh
+++ b/testcases/network/nfs/nfs_stress/nfs_lib.sh
@@ -28,7 +28,10 @@ NFS_PARSE_ARGS_CALLER="$TST_PARSE_ARGS"
TST_OPTS="v:t:$TST_OPTS"
TST_PARSE_ARGS=nfs_parse_args
TST_USAGE=nfs_usage
-TST_NEEDS_TMPDIR=1
+TST_ALL_FILESYSTEMS=1
+TST_SKIP_FILESYSTEMS="exfat"
+TST_MOUNT_DEVICE=1
+TST_FORMAT_DEVICE=1
TST_NEEDS_ROOT=1
TST_NEEDS_CMDS="$TST_NEEDS_CMDS mount exportfs mount.nfs"
TST_SETUP="${TST_SETUP:-nfs_setup}"
@@ -68,7 +71,7 @@ get_remote_dir()
local v="$1"
local n="$2"
- echo "$TST_TMPDIR/$v/$n"
+ echo "$TST_MNTPOINT/$v/$n"
}
nfs_get_remote_path()
@@ -165,6 +168,7 @@ nfs_setup()
local local_dir
local remote_dir
local mount_dir
+ local util_version
if [ "$(stat -f . | grep "Type: nfs")" ]; then
tst_brk TCONF "Cannot run nfs-stress test on mounted NFS"
@@ -178,6 +182,14 @@ nfs_setup()
tst_res TINFO "$(mount.nfs -V)"
+ util_version=$(mount.nfs -V | sed 's/.*nfs-utils \([0-9]\)\..*/\1/')
+ if ! tst_is_int "$util_version"; then
+ tst_brk TBROK "Failed to detect mount.nfs major version"
+ fi
+ if [ "$TST_FS_TYPE" = "tmpfs" ] && [ "$util_version" -lt 2 ]; then
+ tst_brk TCONF "Testing tmpfs requires nfs-utils > 1"
+ fi
+
for i in $VERSION; do
type=$(get_socket_type $n)
tst_res TINFO "setup NFSv$i, socket type $type"
@@ -210,6 +222,7 @@ nfs_cleanup()
grep -q "$local_dir" /proc/mounts && umount $local_dir
n=$(( n + 1 ))
done
+ sleep 1
n=0
for i in $VERSION; do
@@ -219,12 +232,15 @@ nfs_cleanup()
if tst_net_use_netns; then
if test -d $remote_dir; then
exportfs -u *:$remote_dir
+ sleep 1
rm -rf $remote_dir
fi
else
tst_rhost_run -c "test -d $remote_dir && exportfs -u *:$remote_dir"
+ sleep 1
tst_rhost_run -c "test -d $remote_dir && rm -rf $remote_dir"
fi
+
n=$(( n + 1 ))
done
}
--
2.40.0
nfs05_make_tree.c runs make which needs on Btrfs quite a lot of
temporary space. This is a preparation for the next commit which
start using all filesystems via TST_ALL_FILESYSTEMS=1. Currently we use
300 MB, which was not enough for Btrfs:
Filesystem Type Size Used Avail Use% Mounted on
/dev/loop0 btrfs 300M 62M 20K 100% /tmp/LTP_nfs05.Vau10kcszO/mntpoint
After lowering the default values 96% (58M) is being used.
Proper solution would be to detect available size in nfs05_make_tree.c
and lower down values based on free space.
Signed-off-by: Petr Vorel <[email protected]>
---
New in v4.
testcases/network/nfs/nfs_stress/nfs05.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/testcases/network/nfs/nfs_stress/nfs05.sh b/testcases/network/nfs/nfs_stress/nfs05.sh
index c18ef1ab4..34151b67a 100755
--- a/testcases/network/nfs/nfs_stress/nfs05.sh
+++ b/testcases/network/nfs/nfs_stress/nfs05.sh
@@ -8,8 +8,8 @@
#
# Created by: Robbie Williamson ([email protected])
-DIR_NUM=${DIR_NUM:-"10"}
-FILE_NUM=${FILE_NUM:-"30"}
+DIR_NUM=${DIR_NUM:-"8"}
+FILE_NUM=${FILE_NUM:-"28"}
THREAD_NUM=${THREAD_NUM:-"8"}
TST_NEEDS_CMDS="make gcc"
TST_TESTFUNC="do_test"
--
2.40.0
Hi all,
> nfs05_make_tree.c runs make which needs on Btrfs quite a lot of
> temporary space. This is a preparation for the next commit which
> start using all filesystems via TST_ALL_FILESYSTEMS=1. Currently we use
> 300 MB, which was not enough for Btrfs:
Also, based on this detection we could add support for tst_device acquire SIZE
into tst_test.sh to support more than the default 300 MB (as we see the actual
use is obviously less for some filesystems). This would need to check size with
tst_fs_has_free.
Even we don't add this feature, it'd be more friendly to check for space
before creating the loop device:
if ! tst_fs_has_free $TST_TMPDIR 300MB; then
tst_brkm TCONF "Insufficient disk space to create a backing file for loop device"
fi
To be honest, I'd like to merge this patchset first, the rest can wait after the
release.
Kind regards,
Petr
> Filesystem Type Size Used Avail Use% Mounted on
> /dev/loop0 btrfs 300M 62M 20K 100% /tmp/LTP_nfs05.Vau10kcszO/mntpoint
> After lowering the default values 96% (58M) is being used.
> Proper solution would be to detect available size in nfs05_make_tree.c
> and lower down values based on free space.
> Signed-off-by: Petr Vorel <[email protected]>
> ---
> New in v4.
> testcases/network/nfs/nfs_stress/nfs05.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
> diff --git a/testcases/network/nfs/nfs_stress/nfs05.sh b/testcases/network/nfs/nfs_stress/nfs05.sh
> index c18ef1ab4..34151b67a 100755
> --- a/testcases/network/nfs/nfs_stress/nfs05.sh
> +++ b/testcases/network/nfs/nfs_stress/nfs05.sh
> @@ -8,8 +8,8 @@
> # Created by: Robbie Williamson ([email protected])
> -DIR_NUM=${DIR_NUM:-"10"}
> -FILE_NUM=${FILE_NUM:-"30"}
> +DIR_NUM=${DIR_NUM:-"8"}
> +FILE_NUM=${FILE_NUM:-"28"}
> THREAD_NUM=${THREAD_NUM:-"8"}
> TST_NEEDS_CMDS="make gcc"
> TST_TESTFUNC="do_test"
Hi all,
this patch shown also problems on vfat on various new systems (and on older
Debian with nfs-utils 1.3.3 even on all filesystems) - '2' is not visible.
More info at the patch:
https://lore.kernel.org/ltp/[email protected]/
Kind regards,
Petr
Hi all,
> Hi all,
> > nfs05_make_tree.c runs make which needs on Btrfs quite a lot of
> > temporary space. This is a preparation for the next commit which
> > start using all filesystems via TST_ALL_FILESYSTEMS=1. Currently we use
> > 300 MB, which was not enough for Btrfs:
> Also, based on this detection we could add support for tst_device acquire SIZE
> into tst_test.sh to support more than the default 300 MB (as we see the actual
> use is obviously less for some filesystems). This would need to check size with
> tst_fs_has_free.
> Even we don't add this feature, it'd be more friendly to check for space
> before creating the loop device:
> if ! tst_fs_has_free $TST_TMPDIR 300MB; then
> tst_brkm TCONF "Insufficient disk space to create a backing file for loop device"
> fi
> To be honest, I'd like to merge this patchset first, the rest can wait after the
> release.
> Kind regards,
> Petr
> > Filesystem Type Size Used Avail Use% Mounted on
> > /dev/loop0 btrfs 300M 62M 20K 100% /tmp/LTP_nfs05.Vau10kcszO/mntpoint
> > After lowering the default values 96% (58M) is being used.
> > Proper solution would be to detect available size in nfs05_make_tree.c
> > and lower down values based on free space.
> > Signed-off-by: Petr Vorel <[email protected]>
> > ---
> > New in v4.
> > testcases/network/nfs/nfs_stress/nfs05.sh | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> > diff --git a/testcases/network/nfs/nfs_stress/nfs05.sh b/testcases/network/nfs/nfs_stress/nfs05.sh
> > index c18ef1ab4..34151b67a 100755
> > --- a/testcases/network/nfs/nfs_stress/nfs05.sh
> > +++ b/testcases/network/nfs/nfs_stress/nfs05.sh
> > @@ -8,8 +8,8 @@
> > # Created by: Robbie Williamson ([email protected])
> > -DIR_NUM=${DIR_NUM:-"10"}
> > -FILE_NUM=${FILE_NUM:-"30"}
> > +DIR_NUM=${DIR_NUM:-"8"}
> > +FILE_NUM=${FILE_NUM:-"28"}
In the end these values are needed, when testing on systems with 64KB page size:
DIR_NUM=${DIR_NUM:-"5"}
FILE_NUM=${FILE_NUM:-"20"}
THREAD_NUM=${THREAD_NUM:-"5"}
Looking at PAGE_SIZE_LESS_THAN_256KB, CONFIG_PAGE_SIZE_LESS_THAN_256KB (option
for all archs) and checking arch dependent configs I see that there can be 4KB,
8KB, 16KB, 32KB, 64KB, 256KB (each arch allows only some of them). Also
filesystems evolve, thus finding safe value for *all* filesystems on different
page sizes on different kernel versions might be a bit complicated.
Kind regards,
Petr
> > THREAD_NUM=${THREAD_NUM:-"8"}
> > TST_NEEDS_CMDS="make gcc"
> > TST_TESTFUNC="do_test"