Hi,
i seem to have experienced some ext3 inconsistencys - After some reboots
today i was wondering why cron wasnt running and discovered that
starting cron failed because /var/run/crond.pid could not be written.
ls did not show and file under that name. touch showed i/o error on that
file although other file in that directory could be touched.
When i tried to rm crond.pid this showed up:
EXT3-fs warning (device hda8): ext3_unlink: Deleting nonexistent file (107669), 0
After that i could touch the file again and crond did not refuse to start anymore.
I havent experienced ANY ide failures on this disk so far
Some more debugging output:
Nov 14 14:17:12 touch /usr/sbin/cron[621]: (CRON) DEATH (can't open or create /var/run/crond.pid: Input/output error)
touch:/var/run# ls -la
total 96
drwxr-xr-x 13 root root 4096 Nov 14 18:34 .
drwxr-xr-x 18 root root 4096 May 22 13:45 ..
srw-rw-rw- 1 root root 0 Nov 14 17:22 .acpid.socket
-rw-r--r-- 1 root root 4 Nov 14 17:22 atd.pid
drwxr-xr-x 2 root root 4096 Mar 25 2002 autofs
-rw-r--r-- 1 root root 5 Nov 14 18:34 crond.pid
---------- 1 root root 0 Nov 14 18:34 crond.reboot
-rw-r--r-- 1 root root 4 Nov 14 17:22 dhclient.eth0.pid
drwxr-xr-x 2 freerad freerad 4096 Sep 12 19:28 freeradius
-rw-r----- 1 root root 4 Nov 14 17:22 ippl.pid
drwxr-xr-x 2 root root 4096 Nov 6 14:24 iptraf
-rw-r--r-- 1 root root 4 Nov 14 17:23 kdm.pid
-rw-r--r-- 1 root root 4 Nov 14 17:22 klogd.pid
drwxr-xr-x 2 postgres postgres 4096 May 8 2003 postgresql
drwxr-xr-x 2 root root 4096 Nov 14 17:24 samba
drwxrwxr-x 4 root utmp 4096 Aug 24 2002 screen
-rw------- 1 root root 4 Nov 14 17:22 smartd.pid
drwxr-xr-x 2 root root 4096 Jun 24 2002 sshd
-rw-r--r-- 1 root root 4 Nov 14 17:22 sshd.pid
drwx------ 3 root root 4096 Oct 9 14:58 sudo
-rw-r--r-- 1 root root 4 Nov 14 17:22 syslogd.pid
drwx------ 2 root root 4096 Mar 26 2002 usb
-rw-rw-r-- 1 root utmp 4608 Nov 14 17:30 utmp
drwxr-xr-x 2 root root 4096 Nov 14 17:23 xdmctl
drwxr-xr-x 3 root root 4096 Sep 3 19:47 zope
touch:/var/run# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/hda6 489992 100238 364454 22% /
/dev/hda7 3375896 2669948 534456 84% /usr
/dev/hda8 1446780 635796 737492 47% /var
/dev/hda9 972404 18384 904624 2% /tmp
/dev/hda10 32724116 19934576 11127216 65% /home
//dump.mediaways.net/dump
142065664 139608064 2457600 99% /mnt/dump
touch:/var/run# mount
/dev/hda6 on / type ext3 (rw,errors=remount-ro)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/hda7 on /usr type ext3 (rw)
/dev/hda8 on /var type ext3 (rw)
/dev/hda9 on /tmp type ext3 (rw)
/dev/hda10 on /home type ext3 (rw)
usbfs on /proc/bus/usb type usbfs (rw)
//dump.mediaways.net/dump on /mnt/dump type smbfs (rw)
touch:/var/run# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/hda6 126976 10234 116742 9% /
/dev/hda7 429408 125260 304148 30% /usr
/dev/hda8 183936 9772 174164 6% /var
/dev/hda9 123648 53 123595 1% /tmp
/dev/hda10 4161536 111674 4049862 3% /home
//dump.mediaways.net/dump
0 0 0 - /mnt/dump
touch:/var/run# tune2fs -l /dev/hda8
tune2fs 1.34-WIP (21-May-2003)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 4b930ab7-ed5f-4916-90e7-fca4f61e704b
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 183936
Block count: 367479
Reserved block count: 18373
Free blocks: 202934
Free inodes: 174205
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 15328
Inode blocks per group: 479
Filesystem created: Thu May 22 13:02:48 2003
Last mount time: Fri Nov 14 17:22:24 2003
Last write time: Fri Nov 14 17:22:24 2003
Mount count: 49
Maximum mount count: -1
Last checked: Thu May 22 13:02:48 2003
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
First orphan inode: 107669
Default directory hash: tea
Directory Hash Seed: 095c50da-7842-4c7e-9cb5-859a17832ee6
Flo
--
Florian Lohoff [email protected] +49-171-2280134
Heisenberg may have been here.
On Nov 14, 2003 18:42 +0100, Florian Lohoff wrote:
> i seem to have experienced some ext3 inconsistencys - After some reboots
> today i was wondering why cron wasnt running and discovered that
> starting cron failed because /var/run/crond.pid could not be written.
> ls did not show and file under that name. touch showed i/o error on that
> file although other file in that directory could be touched.
>
> When i tried to rm crond.pid this showed up:
>
> EXT3-fs warning (device hda8): ext3_unlink: Deleting nonexistent file (107669), 0
>
> After that i could touch the file again and crond did not refuse to start anymore.
This sounds like the htree "get back deleted entry on directory split" bug
that was fixed months ago in 2.6 htree, but not in any 2.4 patches. Did
you test htree on this system under 2.4 recently?
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
On Fri, Nov 14, 2003 at 10:57:24AM -0700, Andreas Dilger wrote:
> On Nov 14, 2003 18:42 +0100, Florian Lohoff wrote:
> > i seem to have experienced some ext3 inconsistencys - After some reboots
> > today i was wondering why cron wasnt running and discovered that
> > starting cron failed because /var/run/crond.pid could not be written.
> > ls did not show and file under that name. touch showed i/o error on that
> > file although other file in that directory could be touched.
> >
> > When i tried to rm crond.pid this showed up:
> >
> > EXT3-fs warning (device hda8): ext3_unlink: Deleting nonexistent file (107669), 0
> >
> > After that i could touch the file again and crond did not refuse to start anymore.
>
> This sounds like the htree "get back deleted entry on directory split" bug
> that was fixed months ago in 2.6 htree, but not in any 2.4 patches. Did
> you test htree on this system under 2.4 recently?
Nope - I have turned dir_index on on the /tmp filesystem - nowhere else.
Flo
--
Florian Lohoff [email protected] +49-171-2280134
Heisenberg may have been here.