2011-07-20 17:59:20

by Theodore Ts'o

[permalink] [raw]
Subject: Problems building xfsprogs

I was trying to build xfsprogs in preparation for building xfstests in
an hermetic environment (as opposed to depending on magic XFS headers
being installed in /usr/include...)

What I found first of all (building on Ubuntu 10.10) was

a) "make configure" doesn't work until I patch the top-level Makefile:

diff --git a/Makefile b/Makefile
index c40fb2c..37973f5 100644
--- a/Makefile
+++ b/Makefile
@@ -82,7 +82,7 @@ configure:
libtoolize -c $(LIBTOOLIZE_INSTALL) -f
cp include/install-sh .
aclocal -I m4
- autoconf
+ autoconf -I m4

include/builddefs: configure
./configure $$LOCAL_CONFIGURE_OPTIONS

b) By default xfsprogs builds with DEBUG, and this causes building
libxfs/xfs_ialloc.c to blow up since there a reference of
xfs_stack_trace() which is undefined under a #ifdef DEBUG. The way to
fix this is to build with DEBUG=-DNDEBUG, but it took me an hour or
two figure this out....

- Ted


2011-07-20 18:32:20

by Eric Sandeen

[permalink] [raw]
Subject: Re: Problems building xfsprogs

On 07/20/2011 12:59 PM, Ted Ts'o wrote:
> I was trying to build xfsprogs in preparation for building xfstests in
> an hermetic environment (as opposed to depending on magic XFS headers
> being installed in /usr/include...)
>
> What I found first of all (building on Ubuntu 10.10) was
>
> a) "make configure" doesn't work until I patch the top-level Makefile:
>
> diff --git a/Makefile b/Makefile
> index c40fb2c..37973f5 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -82,7 +82,7 @@ configure:
> libtoolize -c $(LIBTOOLIZE_INSTALL) -f
> cp include/install-sh .
> aclocal -I m4
> - autoconf
> + autoconf -I m4
>
> include/builddefs: configure
> ./configure $$LOCAL_CONFIGURE_OPTIONS

How does it fail? Works here(tm):

[sandeen@sandeen tmp]$ git clone
git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
[sandeen@sandeen xfsprogs-dev]$ cd xfsprogs-dev
[sandeen@sandeen xfsprogs-dev]$ make
...
aclocal -I m4
autoconf
./configure $LOCAL_CONFIGURE_OPTIONS
checking build system type... x86_64-redhat-linux-gnu
checking host system type... x86_64-redhat-linux-gnu
...
[CC] attrset.o
[LD] xfs_db
[sandeen@sandeen xfsprogs-dev]$

Wonder what the difference is...

> b) By default xfsprogs builds with DEBUG, and this causes building
> libxfs/xfs_ialloc.c to blow up since there a reference of
> xfs_stack_trace() which is undefined under a #ifdef DEBUG. The way to
> fix this is to build with DEBUG=-DNDEBUG, but it took me an hour or
> two figure this out....

Odd, libxfs/Makefile has:

DEBUG = -DNDEBUG

But to be honest, the DEBUG stuff has been a thorn in my side for Fedora
too, with the extra CFLAGS it sets, something gets goobered up by
default, and I build with -DNDEBUG for those packages too.

I just haven't sorted it out yet ....

So things to look into, but can you also describe how building xfstests
fails for you when you have all the ubuntu xfsprogs-related packages
installed?

-Eric


> - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html


2011-07-20 21:33:24

by Theodore Ts'o

[permalink] [raw]
Subject: Re: Problems building xfsprogs

On Wed, Jul 20, 2011 at 01:32:16PM -0500, Eric Sandeen wrote:
>
> How does it fail? Works here(tm):

It doesn't find the autoconfig macros defined in the m4 directory:

<tytso@tytso-glaptop> {/kbuild/xfstests/xfsprogs-dev} [master]
512% autoconf
autoconf: Undefined macros:
configure.in:100:AC_PACKAGE_NEED_PTHREADMUTEXINIT
configure.in:102:AC_HAVE_FADVISE
configure.in:103:AC_HAVE_MADVISE
configure.in:104:AC_HAVE_MINCORE
configure.in:105:AC_HAVE_SENDFILE
configure.in:106:AC_HAVE_GETMNTENT
configure.in:107:AC_HAVE_GETMNTINFO
configure.in:108:AC_HAVE_FALLOCATE
configure.in:109:AC_HAVE_FIEMAP
configure.in:110:AC_HAVE_BLKID_TOPO($enable_blkid)
configure.in:112:AC_TYPE_PSINT
configure.in:113:AC_TYPE_PSUNSIGNED
configure.in:114:AC_TYPE_U32
configure.in:115:AC_SIZEOF_POINTERS_AND_LONG
configure.in:116:AC_MANUAL_FORMAT
configure.in:3:AC_CONFIG_MACRO_DIR([m4])
configure.in:7:AC_PROG_LIBTOOL
configure.in:89:AC_PACKAGE_GLOBALS(xfsprogs)
configure.in:90:AC_PACKAGE_UTILITIES(xfsprogs)
configure.in:91:AC_MULTILIB($enable_lib64)
configure.in:93:AC_PACKAGE_NEED_AIO_H
configure.in:94:AC_PACKAGE_NEED_LIO_LISTIO
configure.in:96:AC_PACKAGE_NEED_UUID_H
configure.in:97:AC_PACKAGE_NEED_UUIDCOMPARE
configure.in:99:AC_PACKAGE_NEED_PTHREAD_H

Maybe you have all of these autoconf macros stuffed in some
distro-supplied global m4 directory?

> > b) By default xfsprogs builds with DEBUG, and this causes building
> > libxfs/xfs_ialloc.c to blow up since there a reference of
> > xfs_stack_trace() which is undefined under a #ifdef DEBUG. The way to
> > fix this is to build with DEBUG=-DNDEBUG, but it took me an hour or
> > two figure this out....
>
> Odd, libxfs/Makefile has:
>
> DEBUG = -DNDEBUG

Yeah, but include/builddefs{.in}, DEBUG is defined to @debug_build@,
which is -DDEBUG, and then GCFLAGS is set to $(OPTIMIZER) $(DEBUG) ...,

and the override in libxfs/Makefile doesn't change the value of
GCFLAGS, at least not for gmake 3.81.

> But to be honest, the DEBUG stuff has been a thorn in my side for Fedora
> too, with the extra CFLAGS it sets, something gets goobered up by
> default, and I build with -DNDEBUG for those packages too.
>
> I just haven't sorted it out yet ....
>
> So things to look into, but can you also describe how building xfstests
> fails for you when you have all the ubuntu xfsprogs-related packages
> installed?

Yeah, working on it. I'm starting from scratch so I can have a shell
script that can build all of this by hand, instead of manually fixing
these things one at a time as I've done in the past...

- Ted

2011-07-20 22:26:25

by Greg Freemyer

[permalink] [raw]
Subject: Re: Problems building xfsprogs

On Wed, Jul 20, 2011 at 5:33 PM, Ted Ts'o <[email protected]> wrote:
> On Wed, Jul 20, 2011 at 01:32:16PM -0500, Eric Sandeen wrote:
>>
>> How does it fail? ?Works here(tm):
>
> It doesn't find the autoconfig macros defined in the m4 directory:
>
> <tytso@tytso-glaptop> {/kbuild/xfstests/xfsprogs-dev} ?[master]
> 512% autoconf
> autoconf: Undefined macros:
> configure.in:100:AC_PACKAGE_NEED_PTHREADMUTEXINIT
> configure.in:102:AC_HAVE_FADVISE
> configure.in:103:AC_HAVE_MADVISE
> configure.in:104:AC_HAVE_MINCORE
> configure.in:105:AC_HAVE_SENDFILE
> configure.in:106:AC_HAVE_GETMNTENT
> configure.in:107:AC_HAVE_GETMNTINFO
> configure.in:108:AC_HAVE_FALLOCATE
> configure.in:109:AC_HAVE_FIEMAP
> configure.in:110:AC_HAVE_BLKID_TOPO($enable_blkid)
> configure.in:112:AC_TYPE_PSINT
> configure.in:113:AC_TYPE_PSUNSIGNED
> configure.in:114:AC_TYPE_U32
> configure.in:115:AC_SIZEOF_POINTERS_AND_LONG
> configure.in:116:AC_MANUAL_FORMAT
> configure.in:3:AC_CONFIG_MACRO_DIR([m4])
> configure.in:7:AC_PROG_LIBTOOL
> configure.in:89:AC_PACKAGE_GLOBALS(xfsprogs)
> configure.in:90:AC_PACKAGE_UTILITIES(xfsprogs)
> configure.in:91:AC_MULTILIB($enable_lib64)
> configure.in:93:AC_PACKAGE_NEED_AIO_H
> configure.in:94:AC_PACKAGE_NEED_LIO_LISTIO
> configure.in:96:AC_PACKAGE_NEED_UUID_H
> configure.in:97:AC_PACKAGE_NEED_UUIDCOMPARE
> configure.in:99:AC_PACKAGE_NEED_PTHREAD_H
>
> Maybe you have all of these autoconf macros stuffed in some
> distro-supplied global m4 directory?
>
>> > b) By default xfsprogs builds with DEBUG, and this causes building
>> > libxfs/xfs_ialloc.c to blow up since there a reference of
>> > xfs_stack_trace() which is undefined under a #ifdef DEBUG. ?The way to
>> > fix this is to build with DEBUG=-DNDEBUG, but it took me an hour or
>> > two figure this out....
>>
>> Odd, libxfs/Makefile has:
>>
>> DEBUG = -DNDEBUG
>
> Yeah, but include/builddefs{.in}, DEBUG is defined to @debug_build@,
> which is -DDEBUG, and then GCFLAGS is set to $(OPTIMIZER) $(DEBUG) ...,
>
> and the override in libxfs/Makefile doesn't change the value of
> GCFLAGS, at least not for gmake 3.81.
>
>> But to be honest, the DEBUG stuff has been a thorn in my side for Fedora
>> too, with the extra CFLAGS it sets, something gets goobered up by
>> default, and I build with -DNDEBUG for those packages too.
>>
>> I just haven't sorted it out yet ....
>>
>> So things to look into, but can you also describe how building xfstests
>> fails for you when you have all the ubuntu xfsprogs-related packages
>> installed?
>
> Yeah, working on it. ?I'm starting from scratch so I can have a shell
> script that can build all of this by hand, instead of manually fixing
> these things one at a time as I've done in the past...

This little shell script is part of the openSUSE qa test (or soon will
be). ie. Every opensuse distro build is run through automated qa
tests. The testing team is working on adding in xfstests specifically
for btrfs at present.

I don't know exactly what packages are installed when the below runs,
but it should be a fairly basic set of packages since its just a QA
basic functionality run.

===
#!/bin/sh
tmp=/tmp/xfs_playground
export TEST_DEV=/dev/vdb
export TEST_DIR=$tmp/test
export SCRATCH_DEV=/dev/vdc
export SCRATCH_MNT=$tmp/scratch
export FSTYP=btrfs

grep -q susetest /etc/hosts || echo "10.0.2.15 susetest.zq1.de
susetest" >> /etc/hosts # workaround factory bug # TODO: drop later

sudo zypper -n install sudo git gcc libtool automake gettext-tools
libattr-devel libacl-devel libuuid-devel btrfsprogs
mkdir -p $tmp $TEST_DIR $SCRATCH_MNT
cd $tmp

#git clone git://oss.sgi.com/xfs/xfs #(this step may not be needed?)
git clone git://oss.sgi.com/xfs/cmds/xfsprogs
git clone git://oss.sgi.com/xfs/cmds/xfstests

(
cd xfsprogs
make
sudo make install-qa
)
cd xfstests
make


mkfs.$FSTYP $TEST_DEV
mkfs.$FSTYP $SCRATCH_DEV

mount $TEST_DEV $TEST_DIR
mount $SCRATCH_DEV $SCRATCH_MNT

time ./check -g auto -f quick

umount $TEST_DIR
umount $SCRATCH_MNT

echo "xfstests done" > /dev/ttyS0
====

So far the xfstest part of the above seems fine.

It is finding 3 or 4 btrfs failed tests.

Greg

2011-07-21 16:31:27

by Theodore Ts'o

[permalink] [raw]
Subject: Re: Problems building xfsprogs

On Thu, Jul 21, 2011 at 05:06:19AM -0400, Christoph Hellwig wrote:
> I works fine for me on various releases of Debian, Fedsora, RHEL and
> SLES. But all he machines I've quickly checked seem to have an autoconf
> that can handle -I, so could you please send this pach with a proper
> signoff and description?

Sure, will do.

> > b) By default xfsprogs builds with DEBUG, and this causes building
> > libxfs/xfs_ialloc.c to blow up since there a reference of
> > xfs_stack_trace() which is undefined under a #ifdef DEBUG. The way to
> > fix this is to build with DEBUG=-DNDEBUG, but it took me an hour or
> > two figure this out....
>
> The defaul actually is DEBUG=-DNDEBUG. No sure how you go he debug
> build. but we surely should fix it. I'll add the stub as soon as I get
> some time.

Well, the default from xfstest-dev/m4/package_globals.m4 is:

DEBUG=${DEBUG:-'-DDEBUG'} dnl -DNDEBUG
debug_build="$DEBUG"
AC_SUBST(debug_build)

I'm guessing you're getting the default of DEBUG=-DNDEBUG because
you're getting the package_globals.m4 package from some other m4
directory....

- Ted



2011-07-21 04:19:07

by Theodore Ts'o

[permalink] [raw]
Subject: Re: Problems building xfsprogs

Another problem with xfsprogs. "make install" iterates into the
debian subdirectory, for no good reason I can see other than to cause
the build to explode in a million pieces....

- Ted

2011-07-21 04:27:31

by Theodore Ts'o

[permalink] [raw]
Subject: Re: Problems building xfsprogs

On Thu, Jul 21, 2011 at 12:19:03AM -0400, Ted Ts'o wrote:
> Another problem with xfsprogs. "make install" iterates into the
> debian subdirectory, for no good reason I can see other than to cause
> the build to explode in a million pieces....

Sorry, never mind, false alarm.

- Ted

2011-07-21 09:06:19

by Christoph Hellwig

[permalink] [raw]
Subject: Re: Problems building xfsprogs

On Wed, Jul 20, 2011 at 01:59:18PM -0400, Ted Ts'o wrote:
> a) "make configure" doesn't work until I patch the top-level Makefile:
>
> diff --git a/Makefile b/Makefile
> index c40fb2c..37973f5 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -82,7 +82,7 @@ configure:
> libtoolize -c $(LIBTOOLIZE_INSTALL) -f
> cp include/install-sh .
> aclocal -I m4
> - autoconf
> + autoconf -I m4
>
> include/builddefs: configure
> ./configure $$LOCAL_CONFIGURE_OPTIONS

I works fine for me on various releases of Debian, Fedsora, RHEL and
SLES. But all he machines I've quickly checked seem to have an autoconf
that can handle -I, so could you please send this pach with a proper
signoff and description?

> b) By default xfsprogs builds with DEBUG, and this causes building
> libxfs/xfs_ialloc.c to blow up since there a reference of
> xfs_stack_trace() which is undefined under a #ifdef DEBUG. The way to
> fix this is to build with DEBUG=-DNDEBUG, but it took me an hour or
> two figure this out....

The defaul actually is DEBUG=-DNDEBUG. No sure how you go he debug
build. but we surely should fix it. I'll add the stub as soon as I get
some time.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

2011-07-21 22:57:43

by Theodore Ts'o

[permalink] [raw]
Subject: Re: Problems building xfsprogs

On Thu, Jul 21, 2011 at 05:06:19AM -0400, Christoph Hellwig wrote:
>
> I works fine for me on various releases of Debian, Fedsora, RHEL and
> SLES. But all he machines I've quickly checked seem to have an autoconf
> that can handle -I, so could you please send this pach with a proper
> signoff and description?

I figured what the core problem is. There's a subtle and hidden
dependency in xfsprogs for an autoconf version which is 2.50 or above.
In Debian and Ubuntu, if autoconf 2.13 is installed, autoconf will try
to automatically figure out whether the configure.in requires 2.13 or
2.50 (since the autoconf maintainers didn't understand the concept of
"backwards compatibility"). Unfortunately, it got this wrong.

Using an explicit AC_PREREQ(2.50) fixes this. I'll be sending patches
shortly....

- Ted