LinuxLists.cc - [RFC v1 0/1] nvme testsuite runtime optimization

2023-04-19 08:57:42

Subject: [RFC v1 0/1] nvme testsuite runtime optimization

2023-04-19 08:58:01

by Daniel Wagner

[permalink] [raw]

Subject: [RFC v1 1/1] nvme: Limit runtime for verification and limit test image size

The reduce the overall runtime of the testsuite by limiting the runtime
of the verification jobs. We are only interested that some normal I/O is
transferred between host and controller.

Also limit the resource usage of the tests. This allows to run more tiny
VMs and thus more parallel blktests on the same machine.

Signed-off-by: Daniel Wagner <[email protected]>
---
common/xfs | 3 ++-
tests/nvme/004 | 2 +-
tests/nvme/005 | 2 +-
tests/nvme/006 | 2 +-
tests/nvme/007 | 2 +-
tests/nvme/008 | 2 +-
tests/nvme/009 | 2 +-
tests/nvme/010 | 5 +++--
tests/nvme/011 | 5 +++--
tests/nvme/012 | 4 ++--
tests/nvme/013 | 4 ++--
tests/nvme/014 | 10 ++++++++--
tests/nvme/015 | 10 ++++++++--
tests/nvme/017 | 2 +-
tests/nvme/018 | 2 +-
tests/nvme/019 | 2 +-
tests/nvme/020 | 2 +-
tests/nvme/021 | 2 +-
tests/nvme/022 | 2 +-
tests/nvme/023 | 2 +-
tests/nvme/024 | 2 +-
tests/nvme/025 | 2 +-
tests/nvme/026 | 2 +-
tests/nvme/027 | 2 +-
tests/nvme/028 | 2 +-
tests/nvme/029 | 2 +-
tests/nvme/031 | 2 +-
tests/nvme/032 | 4 ++--
tests/nvme/034 | 3 ++-
tests/nvme/035 | 4 ++--
tests/nvme/040 | 4 ++--
tests/nvme/041 | 2 +-
tests/nvme/042 | 2 +-
tests/nvme/043 | 2 +-
tests/nvme/044 | 2 +-
tests/nvme/045 | 2 +-
tests/nvme/047 | 2 +-
tests/nvme/048 | 2 +-
38 files changed, 63 insertions(+), 47 deletions(-)

diff --git a/common/xfs b/common/xfs
index 2c5d96164ac1..ee3033da6ccc 100644
--- a/common/xfs
+++ b/common/xfs
@@ -27,7 +27,8 @@ _xfs_run_fio_verify_io() {

_xfs_mkfs_and_mount "${bdev}" "${mount_dir}" >> "${FULL}" 2>&1

- _run_fio_verify_io --size="$sz" --directory="${mount_dir}/"
+ _run_fio_verify_io --size="$sz" --directory="${mount_dir}/" \
+ --time_based --runtime=5s

umount "${mount_dir}" >> "${FULL}" 2>&1
rm -fr "${mount_dir}"
diff --git a/tests/nvme/004 b/tests/nvme/004
index 9dda538b1ac0..b069d4cc1753 100755
--- a/tests/nvme/004
+++ b/tests/nvme/004
@@ -25,7 +25,7 @@ test() {
local port
port="$(_create_nvmet_port "${nvme_trtype}")"

- truncate -s 1G "$TMPDIR/img"
+ truncate -s 256M "$TMPDIR/img"

local loop_dev
loop_dev="$(losetup -f --show "$TMPDIR/img")"
diff --git a/tests/nvme/005 b/tests/nvme/005
index de567a74a891..18d886f19b06 100755
--- a/tests/nvme/005
+++ b/tests/nvme/005
@@ -24,7 +24,7 @@ test() {
local port
port="$(_create_nvmet_port "${nvme_trtype}")"

- truncate -s 1G "$TMPDIR/img"
+ truncate -s 256M "$TMPDIR/img"

local loop_dev
loop_dev="$(losetup -f --show "$TMPDIR/img")"
diff --git a/tests/nvme/006 b/tests/nvme/006
index d993861c06ba..a757b761e966 100755
--- a/tests/nvme/006
+++ b/tests/nvme/006
@@ -24,7 +24,7 @@ test() {

_setup_nvmet

- truncate -s 1G "$TMPDIR/img"
+ truncate -s 256M "$TMPDIR/img"

loop_dev="$(losetup -f --show "$TMPDIR/img")"

diff --git a/tests/nvme/007 b/tests/nvme/007
index d53100f3ff7b..1468a03e8c3c 100755
--- a/tests/nvme/007
+++ b/tests/nvme/007
@@ -25,7 +25,7 @@ test() {

file_path="${TMPDIR}/img"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/008 b/tests/nvme/008
index 5568fe46e463..2452801e9d7b 100755
--- a/tests/nvme/008
+++ b/tests/nvme/008
@@ -26,7 +26,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

loop_dev="$(losetup -f --show "${file_path}")"

diff --git a/tests/nvme/009 b/tests/nvme/009
index 2814c79164ee..1a3cec39cf0b 100755
--- a/tests/nvme/009
+++ b/tests/nvme/009
@@ -24,7 +24,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/010 b/tests/nvme/010
index b7b1d5188e9b..5831243e414d 100755
--- a/tests/nvme/010
+++ b/tests/nvme/010
@@ -26,7 +26,7 @@ test() {
local file_path="${TMPDIR}/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

loop_dev="$(losetup -f --show "${file_path}")"

@@ -41,7 +41,8 @@ test() {
cat "/sys/block/${nvmedev}n1/uuid"
cat "/sys/block/${nvmedev}n1/wwid"

- _run_fio_verify_io --size=950m --filename="/dev/${nvmedev}n1"
+ _run_fio_verify_io --time_based --runtime=5s \
+ --filename="/dev/${nvmedev}n1"

_nvme_disconnect_subsys "${subsys_name}"

diff --git a/tests/nvme/011 b/tests/nvme/011
index 4bfe9af084e4..efd8728bf1e1 100755
--- a/tests/nvme/011
+++ b/tests/nvme/011
@@ -26,7 +26,7 @@ test() {
local file_path="${TMPDIR}/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
@@ -39,7 +39,8 @@ test() {
cat "/sys/block/${nvmedev}n1/uuid"
cat "/sys/block/${nvmedev}n1/wwid"

- _run_fio_verify_io --size=950m --filename="/dev/${nvmedev}n1"
+ _run_fio_verify_io --time_based --runtime=5s \
+ --filename="/dev/${nvmedev}n1"

_nvme_disconnect_subsys "${subsys_name}"

diff --git a/tests/nvme/012 b/tests/nvme/012
index e60082c2e751..6d275748bd0f 100755
--- a/tests/nvme/012
+++ b/tests/nvme/012
@@ -29,7 +29,7 @@ test() {
local file_path="${TMPDIR}/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

loop_dev="$(losetup -f --show "${file_path}")"

@@ -44,7 +44,7 @@ test() {
cat "/sys/block/${nvmedev}n1/uuid"
cat "/sys/block/${nvmedev}n1/wwid"

- _xfs_run_fio_verify_io "/dev/${nvmedev}n1" "900m"
+ _xfs_run_fio_verify_io "/dev/${nvmedev}n1" "200m"

_nvme_disconnect_subsys "${subsys_name}"

diff --git a/tests/nvme/013 b/tests/nvme/013
index 9d60a7df4577..ffeef71a20df 100755
--- a/tests/nvme/013
+++ b/tests/nvme/013
@@ -28,7 +28,7 @@ test() {

local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
@@ -41,7 +41,7 @@ test() {
cat "/sys/block/${nvmedev}n1/uuid"
cat "/sys/block/${nvmedev}n1/wwid"

- _xfs_run_fio_verify_io "/dev/${nvmedev}n1" "900m"
+ _xfs_run_fio_verify_io "/dev/${nvmedev}n1" "200M"

_nvme_disconnect_subsys "${subsys_name}"

diff --git a/tests/nvme/014 b/tests/nvme/014
index d13cff7921da..88ebf2979cc7 100755
--- a/tests/nvme/014
+++ b/tests/nvme/014
@@ -26,7 +26,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

loop_dev="$(losetup -f --show "${file_path}")"

@@ -41,7 +41,13 @@ test() {
cat "/sys/block/${nvmedev}n1/uuid"
cat "/sys/block/${nvmedev}n1/wwid"

- dd if=/dev/urandom of="/dev/${nvmedev}n1" count=128000 bs=4k status=none
+ local sectors
+ local bs
+ sectors="$(blockdev --getss "/dev/${nvmedev}n1")"
+ bs="$(blockdev --getbsz "/dev/${nvmedev}n1")"
+
+ dd if=/dev/urandom of="/dev/${nvmedev}n1" \
+ count="${sectors}" bs="${bs}" status=none

nvme flush "/dev/${nvmedev}" -n 1

diff --git a/tests/nvme/015 b/tests/nvme/015
index bb52ba2598db..fa174e71a3a6 100755
--- a/tests/nvme/015
+++ b/tests/nvme/015
@@ -25,7 +25,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
@@ -38,7 +38,13 @@ test() {
cat "/sys/block/${nvmedev}n1/uuid"
cat "/sys/block/${nvmedev}n1/wwid"

- dd if=/dev/urandom of="/dev/${nvmedev}n1" count=128000 bs=4k status=none
+ local sectors
+ local bs
+ sectors="$(blockdev --getss "/dev/${nvmedev}n1")"
+ bs="$(blockdev --getbsz "/dev/${nvmedev}n1")"
+
+ dd if=/dev/urandom of="/dev/${nvmedev}n1" \
+ count="${sectors}" bs="${bs}" status=none

nvme flush "/dev/${nvmedev}n1" -n 1

diff --git a/tests/nvme/017 b/tests/nvme/017
index f2a95cf276cb..049204fa19dc 100755
--- a/tests/nvme/017
+++ b/tests/nvme/017
@@ -25,7 +25,7 @@ test() {

file_path="${TMPDIR}/img"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/018 b/tests/nvme/018
index 315e79534348..747e316e893f 100755
--- a/tests/nvme/018
+++ b/tests/nvme/018
@@ -26,7 +26,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/019 b/tests/nvme/019
index 4cb3509a12b2..2e6524423b44 100755
--- a/tests/nvme/019
+++ b/tests/nvme/019
@@ -28,7 +28,7 @@ test() {
local nblk_range="10,10,10,10,10,10,10,10,10,10"
local sblk_range="100,200,300,400,500,600,700,800,900,1000"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

loop_dev="$(losetup -f --show "${file_path}")"

diff --git a/tests/nvme/020 b/tests/nvme/020
index 16fdfcc94918..7d094b614972 100755
--- a/tests/nvme/020
+++ b/tests/nvme/020
@@ -26,7 +26,7 @@ test() {
local nblk_range="10,10,10,10,10,10,10,10,10,10"
local sblk_range="100,200,300,400,500,600,700,800,900,1000"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/021 b/tests/nvme/021
index 6ee0af1fe158..d499148e40a3 100755
--- a/tests/nvme/021
+++ b/tests/nvme/021
@@ -25,7 +25,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/022 b/tests/nvme/022
index 1d76ffa44178..166a88bd0559 100755
--- a/tests/nvme/022
+++ b/tests/nvme/022
@@ -25,7 +25,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/023 b/tests/nvme/023
index b65be07edc38..f4a98d95e7bd 100755
--- a/tests/nvme/023
+++ b/tests/nvme/023
@@ -26,7 +26,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

loop_dev="$(losetup -f --show "${file_path}")"

diff --git a/tests/nvme/024 b/tests/nvme/024
index f756797d6f29..1e952aa9c27f 100755
--- a/tests/nvme/024
+++ b/tests/nvme/024
@@ -25,7 +25,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/025 b/tests/nvme/025
index 941bf36f67fb..f3ba5409be20 100755
--- a/tests/nvme/025
+++ b/tests/nvme/025
@@ -25,7 +25,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/026 b/tests/nvme/026
index c3f06c2a377c..95915c5a4bd9 100755
--- a/tests/nvme/026
+++ b/tests/nvme/026
@@ -25,7 +25,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/027 b/tests/nvme/027
index 0ad663ace811..78176930274d 100755
--- a/tests/nvme/027
+++ b/tests/nvme/027
@@ -25,7 +25,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/028 b/tests/nvme/028
index 7de977a81213..eccdc561d3aa 100755
--- a/tests/nvme/028
+++ b/tests/nvme/028
@@ -25,7 +25,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"91fdba0d-f87b-4c25-b80f-db7be1418b9e"
diff --git a/tests/nvme/029 b/tests/nvme/029
index f8b4cbbb9156..db858cdf595d 100755
--- a/tests/nvme/029
+++ b/tests/nvme/029
@@ -59,7 +59,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 1G "${file_path}"
+ truncate -s 256M "${file_path}"

loop_dev="$(losetup -f --show "${file_path}")"

diff --git a/tests/nvme/031 b/tests/nvme/031
index 4e1798246db1..16dfddfc614b 100755
--- a/tests/nvme/031
+++ b/tests/nvme/031
@@ -33,7 +33,7 @@ test() {

_setup_nvmet

- truncate -s 1G "$TMPDIR/img"
+ truncate -s 256M "$TMPDIR/img"

loop_dev="$(losetup -f --show "$TMPDIR/img")"

diff --git a/tests/nvme/032 b/tests/nvme/032
index 017d4a339971..1a467119a2ff 100755
--- a/tests/nvme/032
+++ b/tests/nvme/032
@@ -38,8 +38,8 @@ test_device() {
sysfs="/sys/bus/pci/devices/${pdev}"

# start fio job
- _run_fio_rand_io --filename="$TEST_DEV" --size=1g \
- --group_reporting &> /dev/null &
+ _run_fio_rand_io --filename="$TEST_DEV" --size=256M \
+ --group_reporting --time_based --runtime=1m > /dev/null &

sleep 5

diff --git a/tests/nvme/034 b/tests/nvme/034
index f92e5e20865b..0da188521c20 100755
--- a/tests/nvme/034
+++ b/tests/nvme/034
@@ -26,7 +26,8 @@ test_device() {
port=$(_nvmet_passthru_target_setup "${subsys}")
nsdev=$(_nvmet_passthru_target_connect "${nvme_trtype}" "${subsys}")

- _run_fio_verify_io --size=950m --filename="${nsdev}"
+ _run_fio_verify_io --size=256M --filename="${nsdev}" \
+ --time_based --runtime=5s

_nvme_disconnect_subsys "${subsys}"
_nvmet_passthru_target_cleanup "${port}" "${subsys}"
diff --git a/tests/nvme/035 b/tests/nvme/035
index d169e351e3d0..356b501a16f9 100755
--- a/tests/nvme/035
+++ b/tests/nvme/035
@@ -17,7 +17,7 @@ requires() {
}

device_requires() {
- _require_test_dev_size_mb 1024
+ _require_test_dev_size_mb 256
}

test_device() {
@@ -32,7 +32,7 @@ test_device() {
port=$(_nvmet_passthru_target_setup "${subsys}")
nsdev=$(_nvmet_passthru_target_connect "${nvme_trtype}" "${subsys}")

- _xfs_run_fio_verify_io "${nsdev}" "900m"
+ _xfs_run_fio_verify_io "${nsdev}" "250m"

_nvme_disconnect_subsys "${subsys}"
_nvmet_passthru_target_cleanup "${port}" "${subsys}"
diff --git a/tests/nvme/040 b/tests/nvme/040
index 04bd726cd309..4c28e8061d40 100755
--- a/tests/nvme/040
+++ b/tests/nvme/040
@@ -25,7 +25,7 @@ test() {
echo "Running ${TEST_NAME}"

_setup_nvmet
- truncate -s 1G "$TMPDIR/img"
+ truncate -s 256M "$TMPDIR/img"
loop_dev="$(losetup -f --show "$TMPDIR/img")"

port="$(_create_nvmet_port "${nvme_trtype}")"
@@ -37,7 +37,7 @@ test() {

# start fio job
echo "starting background fio"
- _run_fio_rand_io --filename="/dev/${nvmedev}n1" --size=1g \
+ _run_fio_rand_io --filename="/dev/${nvmedev}n1" --size=250m \
--group_reporting --ramp_time=5 &> /dev/null &
sleep 5

diff --git a/tests/nvme/041 b/tests/nvme/041
index 03e2dab25918..f2f070d928b4 100755
--- a/tests/nvme/041
+++ b/tests/nvme/041
@@ -44,7 +44,7 @@ test() {

_setup_nvmet

- truncate -s 512M "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"b92842df-a394-44b1-84a4-92ae7d112861"
diff --git a/tests/nvme/042 b/tests/nvme/042
index 4ad726f72f5a..4ecfbd990cc6 100755
--- a/tests/nvme/042
+++ b/tests/nvme/042
@@ -41,7 +41,7 @@ test() {

_setup_nvmet

- truncate -s 512M "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}"
port="$(_create_nvmet_port "${nvme_trtype}")"
diff --git a/tests/nvme/043 b/tests/nvme/043
index c031cecf34a5..51d7d566e64b 100755
--- a/tests/nvme/043
+++ b/tests/nvme/043
@@ -42,7 +42,7 @@ test() {

_setup_nvmet

- truncate -s 512M "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}"
port="$(_create_nvmet_port "${nvme_trtype}")"
diff --git a/tests/nvme/044 b/tests/nvme/044
index f2406ecadf7d..d4397565b5b7 100755
--- a/tests/nvme/044
+++ b/tests/nvme/044
@@ -53,7 +53,7 @@ test() {

_setup_nvmet

- truncate -s 512M "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}"
port="$(_create_nvmet_port "${nvme_trtype}")"
diff --git a/tests/nvme/045 b/tests/nvme/045
index 612e5f168e3c..f89b53a59a3b 100755
--- a/tests/nvme/045
+++ b/tests/nvme/045
@@ -55,7 +55,7 @@ test() {

_setup_nvmet

- truncate -s 512M "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}"
port="$(_create_nvmet_port "${nvme_trtype}")"
diff --git a/tests/nvme/047 b/tests/nvme/047
index d56d3be185b0..54f92fa62c78 100755
--- a/tests/nvme/047
+++ b/tests/nvme/047
@@ -28,7 +28,7 @@ test() {
local file_path="$TMPDIR/img"
local subsys_name="blktests-subsystem-1"

- truncate -s 512M "${file_path}"
+ truncate -s 256M "${file_path}"

loop_dev="$(losetup -f --show "${file_path}")"

diff --git a/tests/nvme/048 b/tests/nvme/048
index b4ee7371eb68..374b0b892d96 100755
--- a/tests/nvme/048
+++ b/tests/nvme/048
@@ -84,7 +84,7 @@ test() {
fi
hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}"

- truncate -s 512M "${file_path}"
+ truncate -s 256M "${file_path}"

_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
"b92842df-a394-44b1-84a4-92ae7d112861"
--
2.40.0

2023-04-19 09:39:47

by Chaitanya Kulkarni

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

On 4/19/23 01:56, Daniel Wagner wrote:
> While testing the fc transport I got a bit tired of wait for the I/O jobs to
> finish. Thus here some runtime optimization.
>
> With a small/slow VM I got following values:
>
> with 'optimizations'
> loop:
> real 4m43.981s
> user 0m17.754s
> sys 2m6.249s
>
> rdma:
> real 2m35.160s
> user 0m6.264s
> sys 0m56.230s
>
> tcp:
> real 2m30.391s
> user 0m5.770s
> sys 0m46.007s
>
> fc:
> real 2m19.738s
> user 0m6.012s
> sys 0m42.201s
>
> base:
> loop:
> real 7m35.061s
> user 0m23.493s
> sys 2m54.866s
>
> rdma:
> real 8m29.347s
> user 0m13.078s
> sys 1m53.158s
>
> tcp:
> real 8m11.357s
> user 0m13.033s
> sys 2m43.156s
>
> fc:
> real 5m46.615s
> user 0m12.819s
> sys 1m46.338s
>
>

Those jobs are meant to be run for at least 1G to establish
confidence on the data set and the system under test since SSDs
are in TBs nowadays and we don't even get anywhere close to that,
with your suggestion we are going even lower ...

we cannot change the dataset size for slow VMs, instead add
a command line argument and pass it to tests e.g.
nvme_verification_size=XXX similar to nvme_trtype but don't change
the default values which we have been testing for years now

Testing is supposed to be time consuming especially verification jobs..

-ck

2023-04-19 10:00:28

by Sagi Grimberg

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

>> While testing the fc transport I got a bit tired of wait for the I/O jobs to
>> finish. Thus here some runtime optimization.
>>
>> With a small/slow VM I got following values:
>>
>> with 'optimizations'
>> loop:
>> real 4m43.981s
>> user 0m17.754s
>> sys 2m6.249s

How come loop is doubling the time with this patch?
ratio is not the same before and after.

>>
>> rdma:
>> real 2m35.160s
>> user 0m6.264s
>> sys 0m56.230s
>>
>> tcp:
>> real 2m30.391s
>> user 0m5.770s
>> sys 0m46.007s
>>
>> fc:
>> real 2m19.738s
>> user 0m6.012s
>> sys 0m42.201s
>>
>> base:
>> loop:
>> real 7m35.061s
>> user 0m23.493s
>> sys 2m54.866s
>>
>> rdma:
>> real 8m29.347s
>> user 0m13.078s
>> sys 1m53.158s
>>
>> tcp:
>> real 8m11.357s
>> user 0m13.033s
>> sys 2m43.156s
>>
>> fc:
>> real 5m46.615s
>> user 0m12.819s
>> sys 1m46.338s
>>
>>
>
> Those jobs are meant to be run for at least 1G to establish
> confidence on the data set and the system under test since SSDs
> are in TBs nowadays and we don't even get anywhere close to that,
> with your suggestion we are going even lower ...

Where does the 1G boundary coming from?

> we cannot change the dataset size for slow VMs, instead add
> a command line argument and pass it to tests e.g.
> nvme_verification_size=XXX similar to nvme_trtype but don't change
> the default values which we have been testing for years now
>
> Testing is supposed to be time consuming especially verification jobs..

I like the idea, but I think it may need to be the other way around.
Have shortest possible runs by default.

2023-04-19 11:19:47

by Daniel Wagner

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

On Wed, Apr 19, 2023 at 12:50:10PM +0300, Sagi Grimberg wrote:
>
> > > While testing the fc transport I got a bit tired of wait for the I/O jobs to
> > > finish. Thus here some runtime optimization.
> > >
> > > With a small/slow VM I got following values:
> > >
> > > with 'optimizations'
> > > loop:
> > > real 4m43.981s
> > > user 0m17.754s
> > > sys 2m6.249s
>
> How come loop is doubling the time with this patch?
> ratio is not the same before and after.

first run was with loop, second one with rdma:

nvme/002 (create many subsystems and test discovery) [not run]
runtime 82.089s ...
nvme_trtype=rdma is not supported in this test

nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [not run]
runtime 39.948s ...
nvme_trtype=rdma is not supported in this test
nvme/017 (create/delete many file-ns and test discovery) [not run]
runtime 40.237s ...

nvme/047 (test different queue types for fabric transports) [passed]
runtime ... 13.580s
nvme/048 (Test queue count changes on reconnect) [passed]
runtime ... 6.287s

82 + 40 + 40 - 14 - 6 = 142. So loop runs additional tests. Hmm, though my
optimization didn't work there...

> > Those jobs are meant to be run for at least 1G to establish
> > confidence on the data set and the system under test since SSDs
> > are in TBs nowadays and we don't even get anywhere close to that,
> > with your suggestion we are going even lower ...
>
> Where does the 1G boundary coming from?

No idea, it just the existing hard coded values. I guess it might be from
efa06fcf3c83 ("loop: test partition scanning") which was the first real test
case (according the logs).

> > we cannot change the dataset size for slow VMs, instead add
> > a command line argument and pass it to tests e.g.
> > nvme_verification_size=XXX similar to nvme_trtype but don't change
> > the default values which we have been testing for years now
> >
> > Testing is supposed to be time consuming especially verification jobs..
>
> I like the idea, but I think it may need to be the other way around.
> Have shortest possible runs by default.

Good point, I'll make it configurable. What is a good small default then? There
are some test cases in loop which allocated a 1M file. That's propably too
small.

2023-04-19 13:21:19

by Sagi Grimberg

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

>>>> While testing the fc transport I got a bit tired of wait for the I/O jobs to
>>>> finish. Thus here some runtime optimization.
>>>>
>>>> With a small/slow VM I got following values:
>>>>
>>>> with 'optimizations'
>>>> loop:
>>>> real 4m43.981s
>>>> user 0m17.754s
>>>> sys 2m6.249s
>>
>> How come loop is doubling the time with this patch?
>> ratio is not the same before and after.
>
> first run was with loop, second one with rdma:
>
> nvme/002 (create many subsystems and test discovery) [not run]
> runtime 82.089s ...
> nvme_trtype=rdma is not supported in this test
>
> nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [not run]
> runtime 39.948s ...
> nvme_trtype=rdma is not supported in this test
> nvme/017 (create/delete many file-ns and test discovery) [not run]
> runtime 40.237s ...
>
> nvme/047 (test different queue types for fabric transports) [passed]
> runtime ... 13.580s
> nvme/048 (Test queue count changes on reconnect) [passed]
> runtime ... 6.287s
>
> 82 + 40 + 40 - 14 - 6 = 142. So loop runs additional tests. Hmm, though my
> optimization didn't work there...

How come loop is 4m+ while the others are 2m+ when before all
were the same timeframe more or less?

>
>>> Those jobs are meant to be run for at least 1G to establish
>>> confidence on the data set and the system under test since SSDs
>>> are in TBs nowadays and we don't even get anywhere close to that,
>>> with your suggestion we are going even lower ...
>>
>> Where does the 1G boundary coming from?
>
> No idea, it just the existing hard coded values. I guess it might be from
> efa06fcf3c83 ("loop: test partition scanning") which was the first real test
> case (according the logs).

Was asking Chaitanya why is 1G considered sufficient vs. other sizes?
Why not 10G? Why not 100M?

2023-04-19 21:12:52

by Chaitanya Kulkarni

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

On 4/19/23 02:50, Sagi Grimberg wrote:
>
>>> While testing the fc transport I got a bit tired of wait for the I/O
>>> jobs to
>>> finish. Thus here some runtime optimization.
>>>
>>> With a small/slow VM I got following values:
>>>
>>> with 'optimizations'
>>>     loop:
>>>       real    4m43.981s
>>>       user    0m17.754s
>>>       sys     2m6.249s
>
> How come loop is doubling the time with this patch?
> ratio is not the same before and after.
>
>>>
>>>     rdma:
>>>       real    2m35.160s
>>>       user    0m6.264s
>>>       sys     0m56.230s
>>>
>>>     tcp:
>>>       real    2m30.391s
>>>       user    0m5.770s
>>>       sys     0m46.007s
>>>
>>>     fc:
>>>       real    2m19.738s
>>>       user    0m6.012s
>>>       sys     0m42.201s
>>>
>>> base:
>>>     loop:
>>>       real    7m35.061s
>>>       user    0m23.493s
>>>       sys     2m54.866s
>>>
>>>     rdma:
>>>       real    8m29.347s
>>>       user    0m13.078s
>>>       sys     1m53.158s
>>>
>>>     tcp:
>>>       real    8m11.357s
>>>       user    0m13.033s
>>>       sys     2m43.156s
>>>
>>>     fc:
>>>       real    5m46.615s
>>>       user    0m12.819s
>>>       sys     1m46.338s
>>>
>>>
>>
>> Those jobs are meant to be run for at least 1G to establish
>> confidence on the data set and the system under test since SSDs
>> are in TBs nowadays and we don't even get anywhere close to that,
>> with your suggestion we are going even lower ...
>
> Where does the 1G boundary coming from?
>

I wrote these testcases 3 times, initially they were the part of
nvme-cli tests7-8 years ago, then nvmftests 7-6 years ago, then they
moved to blktests.

In that time some of the testcases would not fail on with small size
such as less than 512MB especially with verification but they were
in the errors with 1G Hence I kept to be 1G.

Now I don't remember why I didn't use bigger size than 1G
should have documented that somewhere ...

>> we cannot change the dataset size for slow VMs, instead add
>> a command line argument and pass it to tests e.g.
>> nvme_verification_size=XXX similar to nvme_trtype but don't change
>> the default values which we have been testing for years now
>>
>> Testing is supposed to be time consuming especially verification jobs..
>
> I like the idea, but I think it may need to be the other way around.
> Have shortest possible runs by default.

see above..

-ck

2023-04-19 21:17:52

by Chaitanya Kulkarni

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

On 4/19/23 06:15, Sagi Grimberg wrote:
>
>>>>> While testing the fc transport I got a bit tired of wait for the
>>>>> I/O jobs to
>>>>> finish. Thus here some runtime optimization.
>>>>>
>>>>> With a small/slow VM I got following values:
>>>>>
>>>>> with 'optimizations'
>>>>>      loop:
>>>>>        real    4m43.981s
>>>>>        user    0m17.754s
>>>>>        sys     2m6.249s
>>>
>>> How come loop is doubling the time with this patch?
>>> ratio is not the same before and after.
>>
>> first run was with loop, second one with rdma:
>>
>> nvme/002 (create many subsystems and test discovery) [not run]
>>      runtime 82.089s ...
>>      nvme_trtype=rdma is not supported in this test
>>
>> nvme/016 (create/delete many NVMeOF block device-backed ns and test
>> discovery) [not run]
>>      runtime 39.948s ...
>>      nvme_trtype=rdma is not supported in this test
>> nvme/017 (create/delete many file-ns and test discovery) [not run]
>>      runtime 40.237s ...
>>
>> nvme/047 (test different queue types for fabric transports) [passed]
>>      runtime    ... 13.580s
>> nvme/048 (Test queue count changes on reconnect) [passed]
>>      runtime    ... 6.287s
>>
>> 82 + 40 + 40 - 14 - 6 = 142. So loop runs additional tests. Hmm,
>> though my
>> optimization didn't work there...
>
> How come loop is 4m+ while the others are 2m+ when before all
> were the same timeframe more or less?
>
>>
>>>> Those jobs are meant to be run for at least 1G to establish
>>>> confidence on the data set and the system under test since SSDs
>>>> are in TBs nowadays and we don't even get anywhere close to that,
>>>> with your suggestion we are going even lower ...
>>>
>>> Where does the 1G boundary coming from?
>>
>> No idea, it just the existing hard coded values. I guess it might be
>> from
>> efa06fcf3c83 ("loop: test partition scanning") which was the first
>> real test
>> case (according the logs).
>
> Was asking Chaitanya why is 1G considered sufficient vs. other sizes?
> Why not 10G? Why not 100M?

See the earlier response ...

-ck

2023-04-19 21:32:37

by Chaitanya Kulkarni

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

we cannot change the dataset size for slow VMs, instead add
>> a command line argument and pass it to tests e.g.
>> nvme_verification_size=XXX similar to nvme_trtype but don't change
>> the default values which we have been testing for years now
>>
>> Testing is supposed to be time consuming especially verification jobs..
>
> I like the idea, but I think it may need to be the other way around.
> Have shortest possible runs by default.

not everyone is running blktests on the slow vms, so I think it should
be the other way around, the default integration of these testcases
using 1G size in various distros, and it is not a good idea to change
that so everyone else who are not running slow vms who should update
their testscripts ...

-ck

2023-04-20 08:35:23

by Daniel Wagner

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

On Wed, Apr 19, 2023 at 09:11:33PM +0000, Chaitanya Kulkarni wrote:
> >> Those jobs are meant to be run for at least 1G to establish
> >> confidence on the data set and the system under test since SSDs
> >> are in TBs nowadays and we don't even get anywhere close to that,
> >> with your suggestion we are going even lower ...
> >
> > Where does the 1G boundary coming from?
> >
>
> I wrote these testcases 3 times, initially they were the part of
> nvme-cli tests7-8 years ago, then nvmftests 7-6 years ago, then they
> moved to blktests.
>
> In that time some of the testcases would not fail on with small size
> such as less than 512MB especially with verification but they were
> in the errors with 1G Hence I kept to be 1G.
>
> Now I don't remember why I didn't use bigger size than 1G
> should have documented that somewhere ...

Can you remember why you chosed to set the image size to 1G and the io size for
fio to 950m in nvme/012 and nvme/013?

I am testing various image sizes and found that small images e.g in the range of
[4..64]m are passing fine but larger ones like [512-...]M do not (no space
left). Note I've added a calc function which does image size - 1M to leave some
room left.

2023-04-20 08:38:27

by Daniel Wagner

[permalink] [raw]

Subject: Re: [RFC v1 0/1] nvme testsuite runtime optimization

On Thu, Apr 20, 2023 at 10:24:15AM +0200, Daniel Wagner wrote:
> On Wed, Apr 19, 2023 at 09:11:33PM +0000, Chaitanya Kulkarni wrote:
> > >> Those jobs are meant to be run for at least 1G to establish
> > >> confidence on the data set and the system under test since SSDs
> > >> are in TBs nowadays and we don't even get anywhere close to that,
> > >> with your suggestion we are going even lower ...
> > >
> > > Where does the 1G boundary coming from?
> > >
> >
> > I wrote these testcases 3 times, initially they were the part of
> > nvme-cli tests7-8 years ago, then nvmftests 7-6 years ago, then they
> > moved to blktests.
> >
> > In that time some of the testcases would not fail on with small size
> > such as less than 512MB especially with verification but they were
> > in the errors with 1G Hence I kept to be 1G.
> >
> > Now I don't remember why I didn't use bigger size than 1G
> > should have documented that somewhere ...
>
> Can you remember why you chosed to set the image size to 1G and the io size for
> fio to 950m in nvme/012 and nvme/013?

forget it, found a commit message which explains it

e5bd71872b3b ("nvme/012,013,035: change fio I/O size and move size definition place")
[...]
Change fio I/O size of nvme/012,013,035 from 950m to 900m, since recent change
increased the xfs log size and it caused fio failure with I/O size 950m.