2016-07-17 14:05:56

by Theodore Ts'o

[permalink] [raw]
Subject: Refreshed rootfs.img for kvm-xfstests

There is an updated rootfs.img file available at:

https://www.kernel.org/pub/linux/kernel/people/tytso/kvm-xfstests

It includes the latest updates from the xfstests-dev upstream.

There is a new set of more nicely formatted, but more importantly,
better organized and expanded documentation for xfstests-bld,
kvm-xfstests, and gce-xfstests. The new documentation is formatted in
Markdown, and can be browsed starting at:

https://github.com/tytso/xfstests-bld/blob/master/README.md

This release also includes an updated armhf chroot distribution,
armhf_root_fs.tar.gz. This was previously released in the testing
subdirectory, but it is now built as an synchrnoized part of the
kvm-xfstests root_fs.img release process. Very rough instructions for
using this tarball can be found here:

https://github.com/tytso/xfstests-bld/blob/master/Documentation/android-xfstests.md

If there is anyone who is interested and has time to look into
creating an automated test runner for armhf_root_fs.tar.gz, ala
kvm-xfstests and gce-xfstests, but using the Android adb and fastboot
utilities to control the Android device, please let me know. I'd love
to find a volunteer interested in working on this project!

- Ted





2016-07-21 20:19:32

by Eric Whitney

[permalink] [raw]
Subject: Re: Refreshed rootfs.img for kvm-xfstests

* Theodore Ts'o <[email protected]>:
> There is an updated rootfs.img file available at:
>
> https://www.kernel.org/pub/linux/kernel/people/tytso/kvm-xfstests
>
> It includes the latest updates from the xfstests-dev upstream.
>
> There is a new set of more nicely formatted, but more importantly,
> better organized and expanded documentation for xfstests-bld,
> kvm-xfstests, and gce-xfstests. The new documentation is formatted in
> Markdown, and can be browsed starting at:
>
> https://github.com/tytso/xfstests-bld/blob/master/README.md
>
> This release also includes an updated armhf chroot distribution,
> armhf_root_fs.tar.gz. This was previously released in the testing
> subdirectory, but it is now built as an synchrnoized part of the
> kvm-xfstests root_fs.img release process. Very rough instructions for
> using this tarball can be found here:
>
> https://github.com/tytso/xfstests-bld/blob/master/Documentation/android-xfstests.md

I've tested this version of the ARM test appliance using the 4k test config
on a Pandaboard with a 4.7-rc7 kernel, and have discovered a few problems with
the file system image as discussed in today's concall (the upshot was that the
build process might be failing somewhere along the way and producing an
incomplete image):

Several files needed to run the appliance are missing from /root - test-env,
test-conf, /conf (and its contents), and runtests.sh. Copying them in
from xfstests-bld solved that.

/usr/bin/awk points to an executable in Ted's home directory. Relinking to
/usr/bin/gawk fixed things up well enough.

With these changes, I was able to get a complete 4k run with some
discrepancies relative to my ARM baseline:

As reported in the output log produced by runtests.sh, it appears that the
quota user tools are not installed; quota-related xfstests were not run,
and the tools are missing from /root/xfstests/bin where they are found on the
x86_64 image.

Several tests using dm failed involving the error and flakey targets. Some
succeeded in my baseline, and some are new since the baseline. They need
a closer look to see what's going on - there's no immediate evidence the
root fs image is responsible.

Otherwise, this is an easy and convenient way to run the test appliance on
ARM. Thanks!

Eric

>
> If there is anyone who is interested and has time to look into
> creating an automated test runner for armhf_root_fs.tar.gz, ala
> kvm-xfstests and gce-xfstests, but using the Android adb and fastboot
> utilities to control the Android device, please let me know. I'd love
> to find a volunteer interested in working on this project!
>
> - Ted
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

2016-07-23 15:10:30

by Theodore Ts'o

[permalink] [raw]
Subject: Re: Refreshed rootfs.img for kvm-xfstests

On Thu, Jul 21, 2016 at 04:22:39PM -0400, Eric Whitney wrote:
> I've tested this version of the ARM test appliance using the 4k test config
> on a Pandaboard with a 4.7-rc7 kernel, and have discovered a few problems with
> the file system image as discussed in today's concall (the upshot was that the
> build process might be failing somewhere along the way and producing an
> incomplete image):
>
> Several files needed to run the appliance are missing from /root - test-env,
> test-conf, /conf (and its contents), and runtests.sh. Copying them in
> from xfstests-bld solved that.
>
> /usr/bin/awk points to an executable in Ted's home directory. Relinking to
> /usr/bin/gawk fixed things up well enough.

The first was caused a bug in how a rsync'ed the source tree to the
armhf build host. Debian's build systems have a firewall that prevent
git from working correctly, so I have to rsync the source tree up
build host, and this broke the git tree so the automated generation of
the *.ver files broke, which in turn aborted the installation of the
quota tools.

The second problem had to do with how fakechroot (used because I don't
have root on the arm build host) handles symlinks. Fixed by using the
"symlinks" program rewrite the absolute symlinks to relative symlinks.

I've uploaded an updated armmf_root_fs.tar.gz; please give it a try!

Cheers,

- Ted

2016-07-23 23:10:39

by Eric Whitney

[permalink] [raw]
Subject: Re: Refreshed rootfs.img for kvm-xfstests

* Theodore Ts'o <[email protected]>:
> On Thu, Jul 21, 2016 at 04:22:39PM -0400, Eric Whitney wrote:
> > I've tested this version of the ARM test appliance using the 4k test config
> > on a Pandaboard with a 4.7-rc7 kernel, and have discovered a few problems with
> > the file system image as discussed in today's concall (the upshot was that the
> > build process might be failing somewhere along the way and producing an
> > incomplete image):
> >
> > Several files needed to run the appliance are missing from /root - test-env,
> > test-conf, /conf (and its contents), and runtests.sh. Copying them in
> > from xfstests-bld solved that.
> >
> > /usr/bin/awk points to an executable in Ted's home directory. Relinking to
> > /usr/bin/gawk fixed things up well enough.
>
> The first was caused a bug in how a rsync'ed the source tree to the
> armhf build host. Debian's build systems have a firewall that prevent
> git from working correctly, so I have to rsync the source tree up
> build host, and this broke the git tree so the automated generation of
> the *.ver files broke, which in turn aborted the installation of the
> quota tools.
>
> The second problem had to do with how fakechroot (used because I don't
> have root on the arm build host) handles symlinks. Fixed by using the
> "symlinks" program rewrite the absolute symlinks to relative symlinks.
>
> I've uploaded an updated armmf_root_fs.tar.gz; please give it a try!
>
> Cheers,
>
> - Ted

I've tried the image - the awk and quota tools problems are fixed, but the
missing files in root noted above are still missing. I was able to get a
successful 4k run after copying the missing files into the root directory
from the xfstests-bld tree. There are a number of test failures that didn't
occur in my test baseline, much as before, but nothing yet obviously caused
by the test appliance itself.

Thanks,
Eric


2016-07-24 08:18:27

by Theodore Ts'o

[permalink] [raw]
Subject: Re: Refreshed rootfs.img for kvm-xfstests

On Sat, Jul 23, 2016 at 07:06:34PM -0400, Eric Whitney wrote:
> I've tried the image - the awk and quota tools problems are fixed, but the
> missing files in root noted above are still missing.

And that was due to rsync not being available in the build chroot.
I've uploaded a fixed tar.gz file.

Thanks for the feedback! The armhf build driven from an automated
(and now debugged) build script, so we hopefully won't have any issues
moving forward.

- Ted