2023-10-10 19:16:50

by Ritesh Harjani (IBM)

[permalink] [raw]
Subject: [PATCH 1/2] kvm-xfstests: install-kconfig: Use $ARCH-config instead of $KERN_ARCH-config

$KERN_ARCH is used for make arguments. For configs let's use
$ARCH-config. This should not break anything since as of now we only
have arm64-config for which $ARCH and $KERN_ARCH is same.

Signed-off-by: Ritesh Harjani (IBM) <[email protected]>
---
kernel-build/install-kconfig | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel-build/install-kconfig b/kernel-build/install-kconfig
index fc2a49a..f5b2b8e 100755
--- a/kernel-build/install-kconfig
+++ b/kernel-build/install-kconfig
@@ -140,8 +140,8 @@ fi

FILES=("$CONFIG_FN")

-if test -f "$KCONFIG_DIR/$KERN_ARCH-config" ; then
- FILES+=("$KCONFIG_DIR/$KERN_ARCH-config")
+if test -f "$KCONFIG_DIR/$ARCH-config" ; then
+ FILES+=("$KCONFIG_DIR/$ARCH-config")
fi

if test -n "$DO_BLKTESTS" ; then
--
2.41.0


2023-10-10 19:16:54

by Ritesh Harjani (IBM)

[permalink] [raw]
Subject: [PATCH 2/2] kvm-xfstests: Add support for ppc64

These changes adds support for powerpc ppc64le to kvm-xfstests including
building rootfs image.
(Note: this requires latest qemu i.e. "qemu-system-ppc64 -machine help"
should atleast support pseries-8.1)

Here are the cmds which have been tested.

1. ~/src/tools/xfstests-bld/kernel-build/install-kconfig --arch ppc64le
2. ~/src/tools/xfstests-bld/kernel-build/kbuild --arch ppc64le
3. ~/src/tools/xfstests-bld/run-fstests/kvm-xfstests --arch ppc64le \
-N -o "fstesttyp=ext4" -n 4 -r 4096M -c 64k generic/001 -v
4. ./kvm-xfstests -N --kernel ~/src/linux/out-qemu/vmlinux --kernel-arch=ppc64le \
-o "fstesttyp=ext4" -n 4 -r 4096M -c 64k generic/001 -v
5. ./setup-buildchroot --arch=ppc64el && ./build-appliance --chroot=bookworm-ppc64el
6. On a linux build repo, we can directly use
~/src/tools/xfstests-bld/run-fstests/kvm-xfstests -c 4k,64k generic/001
7. On x86 it will default to tcg mode instead of kvm.

<log>
-------------------- Summary report
KERNEL: kernel 6.6.0-rc1-next-20230913-xfstests-dirty #1 SMP Sat Oct 7 13:35:24 IST 2023 ppc64le
CMDLINE: -c 4k,64k generic/001
CPUS: 2
MEM: 1982.81

ext4/4k: 1 tests, 3 seconds
generic/001 Pass 3s
ext4/64k: 1 tests, 5 seconds
generic/001 Pass 4s
Totals: 2 tests, 0 skipped, 0 failures, 0 errors, 7s

Co-developed-by: Ojaswin Mujoo <[email protected]>
Signed-off-by: Ritesh Harjani (IBM) <[email protected]>
---
kernel-build/install-kconfig | 3 ++
kernel-build/kbuild | 9 ++++++
kernel-build/kernel-configs/ppc64le-config | 9 ++++++
run-fstests/kvm-xfstests | 35 +++++++++++++++++-----
run-fstests/util/arch-funcs | 19 ++++++++++++
run-fstests/util/parse_cli | 5 ++++
run-fstests/util/parse_opt_funcs | 3 ++
7 files changed, 76 insertions(+), 7 deletions(-)
create mode 100644 kernel-build/kernel-configs/ppc64le-config

diff --git a/kernel-build/install-kconfig b/kernel-build/install-kconfig
index f5b2b8e..a79e0e6 100755
--- a/kernel-build/install-kconfig
+++ b/kernel-build/install-kconfig
@@ -31,6 +31,9 @@ do
--arm64)
ARCH=arm64
;;
+ --ppc64le)
+ ARCH=ppc64le
+ ;;
--arch)
shift
ARCH=$1
diff --git a/kernel-build/kbuild b/kernel-build/kbuild
index 28b81b4..3d5ccaa 100755
--- a/kernel-build/kbuild
+++ b/kernel-build/kbuild
@@ -52,6 +52,9 @@ do
--arm64)
ARCH=arm64
;;
+ --ppc64le)
+ ARCH=ppc64le
+ ;;
--dpkg)
DO_DPKG=yes
DPKG_EXPLICIT=yes
@@ -104,6 +107,12 @@ case "$ARCH" in
BLD_DIR=$BLD_DIR_X86_64
fi
;;
+ ppc64le)
+ if test -n "$BLD_DIR_PPC64LE" ; then
+ BLD_DIR=$BLD_DIR_PPC64LE
+ fi
+ ;;
+
*)
echo "unknown architecture: $KERN_ARCH"
exit 1
diff --git a/kernel-build/kernel-configs/ppc64le-config b/kernel-build/kernel-configs/ppc64le-config
new file mode 100644
index 0000000..e748d0d
--- /dev/null
+++ b/kernel-build/kernel-configs/ppc64le-config
@@ -0,0 +1,9 @@
+CONFIG_PPC64=y
+CONFIG_CPU_LITTLE_ENDIAN=y
+CONFIG_PAPR_SCM=y
+CONFIG_KVM_BOOK3S_64=y
+CONFIG_KVM_BOOK3S_64_HV=y
+CONFIG_HVC_CONSOLE=y
+CONFIG_HVC_RTAS=y
+CONFIG_HVCS=y
+CONFIG_VIRTIO_CONSOLE=y
diff --git a/run-fstests/kvm-xfstests b/run-fstests/kvm-xfstests
index 2df74c1..f23c9be 100755
--- a/run-fstests/kvm-xfstests
+++ b/run-fstests/kvm-xfstests
@@ -52,16 +52,29 @@ QUIET="quiet loglevel=0"

DOWNLOAD_BASE_URL="https://mirrors.kernel.org/pub/linux/kernel/people/tytso/kvm-xfstests"
if test -z "$EXPLICIT_ROOT_FS" ; then
- ROOT_FS="$(dirname $DIR)/test-appliance/root_fs.img.$ARCH"
+ if [ "$ARCH" == "ppc64le" ]
+ then
+ # root_fs follows debian naming convention ie ppc64le -> ppc64el
+ ROOT_FS="$(dirname $DIR)/test-appliance/root_fs.img.ppc64el"
+ else
+ ROOT_FS="$(dirname $DIR)/test-appliance/root_fs.img.$ARCH"
+ fi
+
if ! test -f "$ROOT_FS" ; then
- ROOT_FS="$(dirname $DIR)/test-appliance/root_fs.img"
+ ROOT_FS="$(dirname $DIR)/test-appliance/root_fs.img"
fi
if ! test -f "$ROOT_FS" ; then
- f=root_fs.img.$ARCH
- ROOT_FS="$(dirname $DIR)/test-appliance/$f"
- echo "Downloading $f..."
- wget -nv --show-progress -O "$ROOT_FS.new" "$DOWNLOAD_BASE_URL/$f"
- mv "$ROOT_FS.new" "$ROOT_FS"
+ if [ "$ARCH" == "ppc64le" ]
+ then
+ # root_fs follows debian naming convention ie ppc64le -> ppc64el
+ f=root_fs.img.ppc64el
+ else
+ f=root_fs.img.$ARCH
+ fi
+ ROOT_FS="$(dirname $DIR)/test-appliance/$f"
+ echo "Downloading $f..."
+ wget -nv --show-progress -O "$ROOT_FS.new" "$DOWNLOAD_BASE_URL/$f"
+ mv "$ROOT_FS.new" "$ROOT_FS"
fi
fi

@@ -206,6 +219,14 @@ case "$ARCH" in
MACHINE_TYPE=virt
CPU_TYPE=max
;;
+ ppc64le)
+ QEMU_ARCH=ppc64
+ ACCEL=kvm:tcg
+ CONSOLE_DEV=hvc0
+ MACHINE_TYPE=pseries,cap-ail-mode-3=off
+ CPU_TYPE=max
+ GDB=""
+ ;;
*)
echo "Unsupported architecture: $ARCH"
exit 1;
diff --git a/run-fstests/util/arch-funcs b/run-fstests/util/arch-funcs
index 3933867..bc7cf47 100644
--- a/run-fstests/util/arch-funcs
+++ b/run-fstests/util/arch-funcs
@@ -21,6 +21,7 @@ function set_my_arch () {
aarch64) MY_ARCH=arm64 ;;
i386|i686) MY_ARCH=i386 ;;
x86_64) MY_ARCH=amd64 ;;
+ ppc64le) MY_ARCH=ppc64le ;;
*) MY_ARCH=$arch ;;
esac
}
@@ -60,6 +61,12 @@ function set_canonicalized_arch () {
GCE_ARCH="X86_64"
KERN_ARCH="x86_64"
;;
+ # setup_buildchroot uses ppc64el
+ ppc64el|ppc64le)
+ ARCH="ppc64le"
+ GCE_ARCH="ppc64le"
+ KERN_ARCH="powerpc"
+ ;;
*)
echo "Architecture $ARCH not supported"
exit 1
@@ -76,6 +83,11 @@ function set_cross_compile ()
CROSS_COMPILE=aarch64-linux-gnu-
fi
;;
+ ppc64le)
+ if test "$MY_ARCH" != ppc64le ; then
+ CROSS_COMPILE=powerpc64le-linux-gnu-
+ fi
+ ;;
esac
}

@@ -133,6 +145,13 @@ function get_kernel_file_info () {
KERNEL_VERSION=$(dpkg -I "$KERNEL" | grep "^ Version: " | \
awk '{print $2}')
;;
+ *"PowerPC"*)
+ KERNEL_ARCH=ppc64le
+ d=$(dirname $(dirname $(dirname $(dirname "$KERNEL"))))
+ if test -f "$d/.git_version" ; then
+ KERNEL_VERSION=$(cat "$d/.git_version")
+ fi
+ ;;
*)
echo "get-kernel-arch $i is not a kernel" >&2
echo "$info"
diff --git a/run-fstests/util/parse_cli b/run-fstests/util/parse_cli
index c7e133f..9dc574b 100644
--- a/run-fstests/util/parse_cli
+++ b/run-fstests/util/parse_cli
@@ -422,6 +422,11 @@ while (( $# >= 1 )); do
ARCH="$1"
EXPLICIT_ARCH=yes
;;
+ --ppc64le)
+ supported_flavors kvm
+ ARCH="ppc64le"
+ EXPLICIT_ARCH=yes
+ ;;
--arm64)
supported_flavors kvm gce
ARCH="arm64"
diff --git a/run-fstests/util/parse_opt_funcs b/run-fstests/util/parse_opt_funcs
index 5457760..83df45a 100644
--- a/run-fstests/util/parse_opt_funcs
+++ b/run-fstests/util/parse_opt_funcs
@@ -25,6 +25,9 @@ find_kernel_to_use()
arm64)
kernel_in_build=arch/arm64/boot/Image
;;
+ ppc64le)
+ kernel_in_build=arch/powerpc/boot/zImage
+ ;;
*)
echo "Unknown architecture in find_kernel_to_use: $ARCH"
exit 1
--
2.41.0

2023-10-12 17:37:04

by Theodore Ts'o

[permalink] [raw]
Subject: Re: [PATCH 1/2] kvm-xfstests: install-kconfig: Use $ARCH-config instead of $KERN_ARCH-config

On Wed, Oct 11, 2023 at 12:46:30AM +0530, Ritesh Harjani (IBM) wrote:
> $KERN_ARCH is used for make arguments. For configs let's use
> $ARCH-config. This should not break anything since as of now we only
> have arm64-config for which $ARCH and $KERN_ARCH is same.
>
> Signed-off-by: Ritesh Harjani (IBM) <[email protected]>

It would be much better to teach set_canoncalized_arch about the
powerpc64le architecture. That way other scripts that use
set_canoicalized_arch can correctly depend on KERN_ARCH. See
run-fstests/util/arch_funcs:

# There are multiple ways that CPU architectured can be named.
# KERN_ARCH is what is used when specifying ARCH=xxx when building the
# kernel.
#
# GCE_ARCH is what gets used when setting specifying the architecture
# when creating the GCE image using the --architecture flag
# ARCH (after being canonicalized by this function) is defined by
# the Debian architecture port names[1] and is used internally
# by gce-xfstests and kvm-xfstests. So for example, when we add
# support for support for Power architectures, the ARCH name that should
# be used is ppc64 or ppc64el.

...

# This function takes as input a user-supplied architecture (which
# generally should be a Debian port name, but users might use
# a $(uname -m) instead.
#
function set_canonicalized_arch () {
case "$1" in
arm64|aarch64)
ARCH="arm64"
GCE_ARCH="ARM64"
KERN_ARCH="arm64"
;;
...

So basically, after calling set_canoncalized_arch, ARCH should be
Debian architecture port name, GCE_ARCH (if not the empty string) is
the name of the GCE architecture name, and KERN_ARCH should be what
should be passed to the Kernel makefile as "make ARCH=$KERN_ARCH ..."

- Ted

2023-10-23 02:34:37

by Theodore Ts'o

[permalink] [raw]
Subject: New archtecture support in xfstests-bld

Hey Ritesh,

I just pulled some changes from Eric Biggers into xfstests-bld which
has a start on adding riscv64 support to kvm-xfstests. So far, he's
updated libaio to a newer upstream version (newer is relative; the
"new" version was last updated six years ago :-) for better RiscV
support, and he's added RiscV support to set_canonicalized_arch().

I'd recommend that you start with the latest upstream version of
xfstests-bld, and then add support for powerpcle64 by adding support
to find_kernel_to_use() in run=fstests/util/parse_opt_funcs, and
set_cross_compile() and get_kernel_file_info() in
run-fstests/util-arch, since those changes in the 2/2 patch series[1]
were clearly correct. (And Eric, you should take a look at those
changes[1] for RiscV support.)

[1] https://lore.kernel.org/all/eb1f8f0fb0ff9a6358129a2a45bd0c88421ac669.1696965271.git.ritesh.list@gmail.com/

I'd also encourage you to add support for the new architectures in
selftests/appliance, since that will exercise building a kernel for
the foreign architecture using cross-compilation, and then using
qemu-system-$ARCH from kvm-xfstests.

(Yes, kvm-xfstests is starting to get very much misnamed; but kvm is
easier to type, and autocompletes much more nicely than qemu-<tab>.
The string "kvm" also is sprinkled all over the xfstests-bld scripts,
and I'm not convinced it's worth changing. That being said, I've
added a qemu-xfstests script which gets installed into the user's bin
directory; we'll see if that is something people feel strongly about
using the new name.)

Finally, since have two separate efforts to add support for new
architectures to xfstests-bld, might I prevail on you to keep some
notes about what's needed to bootstrap a new architecture? That might
be helpful for some future developer.

Many thanks!

- Ted