Date: Wed, 29 May 2013 07:32:48 +1000
From: Dave Chinner <david@fromorbit.com>
To: Dave Jones <davej@redhat.com>, xfs@oss.sgi.com,
        Linux Kernel <linux-kernel@vger.kernel.org>
Subject: Re: 3.10-rc3 xfs mount/recovery failure & ext fsck hang.
Message-ID: <20130528213248.GC29338@dastard>
References: <20130528161230.GA7577@redhat.com>
 <20130528211012.GX29466@dastard>
 <20130528211544.GB24342@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20130528211544.GB24342@redhat.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2264
Lines: 52

On Tue, May 28, 2013 at 05:15:44PM -0400, Dave Jones wrote:
> On Wed, May 29, 2013 at 07:10:12AM +1000, Dave Chinner wrote:
>  > On Tue, May 28, 2013 at 12:12:30PM -0400, Dave Jones wrote:
>  > > box crashed, and needed rebooting. On next bootup, when it found the dirty partition,
>  > > xfs chose to spew and then hang instead of replaying the journal and mounting :(
>  > > 
>  > > [   14.694731] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, debug enabled
>  > > [   14.722328] XFS (sda2): Mounting Filesystem
>  > > [   14.757801] XFS (sda2): Starting recovery (logdev: internal)
>  > > [   14.782049] XFS: Assertion failed: fs_is_ok, file: fs/xfs/xfs_dir2_data.c, line: 169
>  > 
>  > A directory block has an entry that is not in the hash index.
>  > Either there's an underlying corruption on disk, or there's an
>  > inconsistency in what has been logged and so an entire change has
>  > not been replayed. Hence the post recovery verification has thrown a
>  > corruption error....
>  > 
>  > If you haven't already repaired the filesystem, can you send me a
>  > metadump of the filesystem in question?
> 
> Sorry, too late. If I can repro, I'll do so next time.
> FYI, I ran xfs_repair and it just hung. Wouldn't even answer ctrl-c.
> Rebooted, and then it mounted and recovered just fine!

Strange. I can't think of any reason outside a kernel problem for
xfs_repair going into an uninterruptible sleep. Did it happen after
the repair completed (i.e. after phase 7)? If so, then closing the
block device might have tripped the same problem that fsck.ext2
hit....

>  > > [   40.642521] BUG: soft lockup - CPU#0 stuck for 22s! [fsck.ext2:294]
>  > 
>  > I'm not sure what this has to do with the XFS problem - it's
>  > apparently stuck in invalidate_bh_lrus() walking a CPU mask....
> 
> there for completion, it was sandwiched between the other xfs bits :)

*nod*

You never know what might be relevant to the problem :)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/