2009-02-01 20:50:26

by Andrew Morton

[permalink] [raw]
Subject: Re: [Bugme-new] [Bug 12594] New: filesystem corruption while running bonnie


(switched to email. Please respond via emailed reply-to-all, not via the
bugzilla web interface).

On Sun, 1 Feb 2009 06:59:52 -0800 (PST) [email protected] wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=12594
>
> Summary: filesystem corruption while running bonnie
> Product: File System
> Version: 2.5
> KernelVersion: 2.6.28.1
> Platform: All
> OS/Version: Linux
> Tree: Mainline
> Status: NEW
> Severity: normal
> Priority: P1
> Component: ext3
> AssignedTo: [email protected]
> ReportedBy: [email protected]
>
>
> mkdir /mnt/tmp /tmp/tmp ; mount -t tmpfs /dev/null /mnt
> chown folkert /mnt/tmp /tmp/tmp
>
> and then in parallel:
> while [ 1 ] ; do bonnie++ -d /tmp/tmp -s 128 -r 64 -x 1 -u folkert ; done
> while [ 1 ] ; do bonnie++ -d /mnt/tmp -s 128 -r 64 -x 1 -u folkert ; done
>
> then after say 15 minutes i run the reboot command (normal via init) during the
> fsck (I always do a touch /forcefsck) I get:
> fsck 1.41.3 (12-Oct-2008)
> /dev/mapper/VGODR-tmp contains a file system with errors, check forced.
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (1) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (1) not
> referenced
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (2) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (2) not
> referenced
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (3) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (3) not
> referenced
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (4) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (4) not
> referenced
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (5) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (5) not
> referenced
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (6) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (6) not
> referenced
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (7) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (7) not
> referenced
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (8) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (8) has bad
> max hash
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (8) not
> referenced
> ...
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (523) not
> referenced
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (524) has
> invalid depth (2)
> /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (524) not
> referenced
> /dev/mapper/VGODR-tmp: Invalid HTREE directory inode 69634 (/tmp/Bonnie.3813).
>
> I've got a gzip'ed dump of /tmp available on request.

htree went bad. It's strange that exercising tmpfs would do this.

I always get worried when I see device-mapper in the picture. It has a
history.. Perhaps you can describe your device-mapper setup?



2009-02-04 12:11:25

by folkert

[permalink] [raw]
Subject: Re: [Bugme-new] [Bug 12594] New: filesystem corruption while running bonnie

> > http://bugzilla.kernel.org/show_bug.cgi?id=12594
> >
> > Summary: filesystem corruption while running bonnie
> > Product: File System
> > Version: 2.5
> > KernelVersion: 2.6.28.1
> > Platform: All
> > OS/Version: Linux
> > Tree: Mainline
> > Status: NEW
> > Severity: normal
> > Priority: P1
> > Component: ext3
> > AssignedTo: [email protected]
> > ReportedBy: [email protected]
> >
> >
> > mkdir /mnt/tmp /tmp/tmp ; mount -t tmpfs /dev/null /mnt
> > chown folkert /mnt/tmp /tmp/tmp
> >
> > and then in parallel:
> > while [ 1 ] ; do bonnie++ -d /tmp/tmp -s 128 -r 64 -x 1 -u folkert ; done
> > while [ 1 ] ; do bonnie++ -d /mnt/tmp -s 128 -r 64 -x 1 -u folkert ; done
> >
> > then after say 15 minutes i run the reboot command (normal via init) during the
> > fsck (I always do a touch /forcefsck) I get:
> > fsck 1.41.3 (12-Oct-2008)
> > /dev/mapper/VGODR-tmp contains a file system with errors, check forced.
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (1) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (1) not
> > referenced
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (2) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (2) not
> > referenced
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (3) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (3) not
> > referenced
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (4) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (4) not
> > referenced
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (5) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (5) not
> > referenced
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (6) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (6) not
> > referenced
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (7) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (7) not
> > referenced
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (8) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (8) has bad
> > max hash
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (8) not
> > referenced
> > ...
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (523) not
> > referenced
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (524) has
> > invalid depth (2)
> > /dev/mapper/VGODR-tmp: Problem in HTREE directory inode 69634: node (524) not
> > referenced
> > /dev/mapper/VGODR-tmp: Invalid HTREE directory inode 69634 (/tmp/Bonnie.3813).
> >
> > I've got a gzip'ed dump of /tmp available on request.
>
> htree went bad. It's strange that exercising tmpfs would do this.
>
> I always get worried when I see device-mapper in the picture. It has a
> history.. Perhaps you can describe your device-mapper setup?

(Sorry for the delayed response, had the flu)

The system is an LVM setup with /home, /, /tmp, /var, /usr and swap in
seperate LVM slices.


Folkert van Heusden

--
http://www.ishetweekend.nl
----------------------------------------------------------------------
Phone: +31-6-41278122, PGP-key: 1F28D8AE, http://www.vanheusden.com

2009-02-05 20:00:31

by folkert

[permalink] [raw]
Subject: Re: [Bugme-new] [Bug 12594] New: filesystem corruption while running bonnie

> > htree went bad. It's strange that exercising tmpfs would do this.
> >
> > I always get worried when I see device-mapper in the picture. It has a
> > history.. Perhaps you can describe your device-mapper setup?
>
> (Sorry for the delayed response, had the flu)
>
> The system is an LVM setup with /home, /, /tmp, /var, /usr and swap in
> seperate LVM slices.

Some more details:
odr:/home/folkert# vgdisplay
--- Volume group ---
VG Name VGODR
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 7
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 6
Open LV 6
Max PV 0
Cur PV 1
Act PV 1
VG Size 37.03 GB
PE Size 4.00 MB
Total PE 9479
Alloc PE / Size 9479 / 37.03 GB
Free PE / Size 0 / 0
VG UUID dlQony-04VM-KDwW-2Qul-c3gG-qr6L-faKDMH


odr:/home/folkert# lvdisplay
--- Logical volume ---
LV Name /dev/VGODR/root
VG Name VGODR
LV UUID dpFEsy-myWC-YBhp-bw1t-lakV-rKwe-Pu2lDL
LV Write Access read/write
LV Status available
# open 1
LV Size 276.00 MB
Current LE 69
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:0

--- Logical volume ---
LV Name /dev/VGODR/usr
VG Name VGODR
LV UUID xJdCNc-k3iv-czxs-rUdd-tIi1-0Vab-Xk13U3
LV Write Access read/write
LV Status available
# open 1
LV Size 4.77 GB
Current LE 1221
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:1

--- Logical volume ---
LV Name /dev/VGODR/var
VG Name VGODR
LV UUID lbqzC7-NDIC-t9rT-Eqo2-1iQS-EadB-tnHkPq
LV Write Access read/write
LV Status available
# open 1
LV Size 2.86 GB
Current LE 733
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:2

--- Logical volume ---
LV Name /dev/VGODR/swap_1
VG Name VGODR
LV UUID olCdBr-asFX-3K32-zXQ8-6ee0-ou0M-PYbnro
LV Write Access read/write
LV Status available
# open 2
LV Size 1.38 GB
Current LE 354
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:3

--- Logical volume ---
LV Name /dev/VGODR/tmp
VG Name VGODR
LV UUID nsFpW0-eUtE-F8nL-a4rN-80vQ-6mo2-EVbtph
LV Write Access read/write
LV Status available
# open 1
LV Size 392.00 MB
Current LE 98
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:4

--- Logical volume ---
LV Name /dev/VGODR/home
VG Name VGODR
LV UUID 0l3HlK-rrsR-WWLZ-ZMip-bmHE-tRD2-sp7DAk
LV Write Access read/write
LV Status available
# open 1
LV Size 27.36 GB
Current LE 7004
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:5



Folkert van Heusden

--
MultiTail is een flexibele tool voor het volgen van logfiles en
uitvoer van commando's. Filteren, van kleur voorzien, mergen,
'diff-view', etc. http://www.vanheusden.com/multitail/
----------------------------------------------------------------------
Phone: +31-6-41278122, PGP-key: 1F28D8AE, http://www.vanheusden.com