Hi,
When i just run a test on Centos 6.2 as follows:
################################
#!/bin/bash
while true
do
file="/tmp/filetest"
echo $file
dd if=/dev/zero of=${file} bs=512 count=204800 &> /dev/null
sleep 5
done
################################
the inactive memory keep growing:
#cat /proc/meminfo | grep Inactive\(fi
Inactive(file): 420144 kB
...
#cat /proc/meminfo | grep Inactive\(fi
Inactive(file): 554662 kB
...
#cat /proc/meminfo | grep Inactive\(fi
Inactive(file): 882824 kB
and i cannot reclaim it:
# cat /proc/meminfo | grep Inactive\(file\);
Inactive(file): 882824 kB
# sync;
# echo 3 > /proc/sys/vm/drop_caches
# cat /proc/meminfo | grep Inactive\(file\);
Inactive(file): 777664 kB
My file system type is ext3, mounted with the option data=journal and
it is easy to reproduce. If the mount option is data=ordered,
reproducing this problom need more time. Because of the problom, when
in the final situation, i cannot kmalloc a larger contiguous memory,
especially in interrupt context. I have tested on other version
kernel, such as 2.6.30 and 3.6.11, the problom also exists.
Is this a bug?
Or can you give some tips to avoid this?
--
Regards,
Lenky
Hi, all
I install Ubuntu 11.04 on Dell-Latitude-D630.
Linux kernel is 2.6.39.4(I compiled)
uname -a
Linux Dell-Latitude-D630 2.6.39.4-zhuyj26394 #2 SMP Tue Mar 12 13:23:06
CST 2013 i686 i686 i386 GNU/Linux
I followed these steps:
****************************Begin***************************************
1. get ltp-full-20130109.bz2 and run "./configure;make", then we will
get fsstress;
2. mount -t tmpfs -o size=512m tmpfs /tmp
3. dd if=/dev/zero of=/tmp/img bs=1024 count=$((500*1024))
4. ls -lah /tmp/img
total 501M
drwxrwxrwt 2 root root 60 2013-03-12 15:18 .
drwxr-xr-x 23 root root 4.0K 2013-03-12 13:09 ..
-rw-r--r-- 1 root root 500M 2013-03-12 15:18 img
5. df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 113317544 6994416 100566868 7% /
none 1023772 696 1023076 1% /dev
none 1030440 80 1030360 1% /dev/shm
none 1030440 92 1030348 1% /var/run
none 1030440 0 1030440 0% /var/lock
tmpfs 524288 512504 11784 98% /tmp
6. mkfs.ext4 /tmp/img
mke2fs 1.41.14 (22-Dec-2010)
/tmp/img is not a block special device.
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
128016 inodes, 512000 blocks
25600 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
63 block groups
8192 blocks per group, 8192 fragments per group
2032 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 30 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
7. mkdir /tmp/mnt
8. mount -t ext4 -o loop /tmp/img /tmp/mnt
9. df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 113317544 6994424 100566860 7% /
none 1023772 696 1023076 1% /dev
none 1030440 80 1030360 1% /dev/shm
none 1030440 92 1030348 1% /var/run
none 1030440 0 1030440 0% /var/lock
tmpfs 524288 512504 11784 98% /tmp
/dev/loop0 495844 10510 459734 3% /tmp/mnt
10. cd /tmp/mnt; mkdir /tmp/mnt/tmp1 /tmp/mnt/tmp2; cp ~/fsstress /tmp/mnt
./fsstress -c -p 5 -n 10000 -d . -l 0 -s 10M &
./fsstress -c -p 5 -n 10000 -d ./tmp1 -l 0 -s 10M &
./fsstress -c -p 5 -n 10000 -d ./tmp2 -l 0 -s 10M &
11.when the following appears, run "ps;killall fsstress;ps"
./tmp1: No such file or directory
./tmp2: No such file or directory
[2]- Exit 1 ./fsstress -c -p 5 -n 10000 -d ./tmp1 -l 0
-s 10M
[3]+ Exit 1 ./fsstress -c -p 5 -n 10000 -d ./tmp2 -l 0
-s 10M
These messages will appear:
ps;killall fsstress;ps
PID TTY TIME CMD
1366 pts/0 00:00:01 bash
1600 pts/0 00:00:00 fsstress
1633 pts/0 00:00:07 fsstress
1634 pts/0 00:00:06 fsstress
1635 pts/0 00:00:06 fsstress
1636 pts/0 00:00:07 fsstress
1637 pts/0 00:00:07 fsstress
1640 pts/0 00:00:00 ps
[1]+ Terminated ./fsstress -c -p 5 -n 10000 -d . -l 0 -s 10M
PID TTY TIME CMD
1366 pts/0 00:00:01 bash
1642 pts/0 00:00:00 ps
12. /bin/sync;/bin/echo 1 > /proc/sys/vm/drop_caches
13. rm -Rf p*;rm -Rf tmp*
rm: cannot remove `p0': Directory not empty
rm: cannot remove `p1/d99d/d9b4/da40/da71/da74': Directory not empty
rm: cannot remove `p2/dc/d79/f56f': Input/output error
rm: cannot remove `p2/dc/d79/f362': Input/output error
rm: cannot remove `p2/dc/d79/c460': Input/output error
rm: cannot remove `p2/dc/d79/cf4': Input/output error
rm: cannot remove `p2/dc/d79/f3b0': Input/output error
rm: cannot remove `p2/dc/d79/d107/d95e/d7f7/fb50': Input/output error
rm: cannot remove `p2/dc/d79/d107/d95e/f99e': Input/output error
rm: cannot remove `p2/dc/d79/d107/d95e/fc40': Input/output error
rm: cannot remove `p2/dc/d79/d107/f7ab': Input/output error
rm: cannot remove `p2/dc/d79/d107/d547/c843': Input/output error
rm: cannot remove `p2/dc/d79/d107/d547/c871': Input/output error
rm: cannot remove `p2/dc/d79/d107/fa77': Input/output error
rm: cannot remove `p2/dc/d79/f12e': Input/output error
rm: cannot remove `p2/dc/d79/c623': Input/output error
rm: cannot remove `p2/dc/d79/f207': Input/output error
rm: cannot remove `p2/dc/d79/c216': Input/output error
rm: cannot remove `p2/dc/d79/f23d': Input/output error
rm: cannot remove `p2/dc/d79/c250': Input/output error
rm: cannot remove `p2/dc/d79/f274': Input/output error
rm: cannot remove `p2/dc/d79/f285': Input/output error
rm: cannot remove `p2/dc/d79/f2af': Input/output error
rm: cannot remove `p2/dc/d79/c2d1': Input/output error
rm: cannot remove `p2/dc/d79/c2ee': Input/output error
rm: cannot remove `p2/dc/d79/c35a': Input/output error
.....
14. run "dmesg"
[ 469.002632] EXT4-fs warning (device loop0): dx_probe:364:
Unrecognised inode hash code 12
[ 469.002647] EXT4-fs warning (device loop0): dx_probe:472: Corrupt dir
inode 8423, running e2fsck is recommended.
[ 469.018830] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 8926
[ 469.019792] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 8816
[ 469.019950] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 8570
[ 469.020729] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 7814
[ 469.021227] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 9259
[ 469.048610] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#10178: comm rm: deleted inode referenced: 7791
[ 469.049919] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#10177: comm rm: deleted inode referenced: 8100
[ 469.053826] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#10177: comm rm: deleted inode referenced: 8822
[ 469.060481] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8417: comm rm: deleted inode referenced: 8113
[ 469.062336] EXT4-fs warning (device loop0): dx_probe:364:
Unrecognised inode hash code 12
[ 469.062349] EXT4-fs warning (device loop0): dx_probe:472: Corrupt dir
inode 10163, running e2fsck is recommended.
[ 469.065721] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#10163: comm rm: deleted inode referenced: 9027
[ 469.066138] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#10163: comm rm: deleted inode referenced: 9090
[ 469.067526] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8417: comm rm: deleted inode referenced: 8443
[ 469.068584] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 8067
[ 469.069280] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 9694
[ 469.069893] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 8100
[ 469.070123] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 8635
[ 469.070509] EXT4-fs error (device loop0): ext4_lookup:1044: inode
#8423: comm rm: deleted inode referenced: 8929
......
***************************End**********************************************
But if I use ext3, this will not occur.
If I do not run "/bin/sync;/bin/echo 1 > /proc/sys/vm/drop_caches", this
will not occur.
If I use kernel>=3.0, this will not occur.
But I use ext4 and run "/bin/sync; /bin/echo 1 >
/proc/sys/vm/drop_caches", these error messages.
So is this is a bug?
Hi, all
Almost 2.6.3x has this problem. Is it a problem?
Any reply is appreciated.
Zhuyj
On 03/12/2013 03:41 PM, zhuyj wrote:
> Hi, all
>
> I install Ubuntu 11.04 on Dell-Latitude-D630.
> Linux kernel is 2.6.39.4(I compiled)
>
> uname -a
> Linux Dell-Latitude-D630 2.6.39.4-zhuyj26394 #2 SMP Tue Mar 12
> 13:23:06 CST 2013 i686 i686 i386 GNU/Linux
>
> I followed these steps:
>
> ****************************Begin***************************************
> 1. get ltp-full-20130109.bz2 and run "./configure;make", then we will
> get fsstress;
>
> 2. mount -t tmpfs -o size=512m tmpfs /tmp
>
> 3. dd if=/dev/zero of=/tmp/img bs=1024 count=$((500*1024))
>
> 4. ls -lah /tmp/img
>
> total 501M
> drwxrwxrwt 2 root root 60 2013-03-12 15:18 .
> drwxr-xr-x 23 root root 4.0K 2013-03-12 13:09 ..
> -rw-r--r-- 1 root root 500M 2013-03-12 15:18 img
>
> 5. df
>
> Filesystem 1K-blocks Used Available Use% Mounted on
> /dev/sda1 113317544 6994416 100566868 7% /
> none 1023772 696 1023076 1% /dev
> none 1030440 80 1030360 1% /dev/shm
> none 1030440 92 1030348 1% /var/run
> none 1030440 0 1030440 0% /var/lock
> tmpfs 524288 512504 11784 98% /tmp
>
> 6. mkfs.ext4 /tmp/img
>
> mke2fs 1.41.14 (22-Dec-2010)
> /tmp/img is not a block special device.
> Proceed anyway? (y,n) y
> Filesystem label=
> OS type: Linux
> Block size=1024 (log=0)
> Fragment size=1024 (log=0)
> Stride=0 blocks, Stripe width=0 blocks
> 128016 inodes, 512000 blocks
> 25600 blocks (5.00%) reserved for the super user
> First data block=1
> Maximum filesystem blocks=67633152
> 63 block groups
> 8192 blocks per group, 8192 fragments per group
> 2032 inodes per group
> Superblock backups stored on blocks:
> 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
>
> Writing inode tables: done
> Creating journal (8192 blocks): done
> Writing superblocks and filesystem accounting information: done
>
> This filesystem will be automatically checked every 30 mounts or
> 180 days, whichever comes first. Use tune2fs -c or -i to override.
>
> 7. mkdir /tmp/mnt
>
> 8. mount -t ext4 -o loop /tmp/img /tmp/mnt
>
> 9. df
> Filesystem 1K-blocks Used Available Use% Mounted on
> /dev/sda1 113317544 6994424 100566860 7% /
> none 1023772 696 1023076 1% /dev
> none 1030440 80 1030360 1% /dev/shm
> none 1030440 92 1030348 1% /var/run
> none 1030440 0 1030440 0% /var/lock
> tmpfs 524288 512504 11784 98% /tmp
> /dev/loop0 495844 10510 459734 3% /tmp/mnt
>
> 10. cd /tmp/mnt; mkdir /tmp/mnt/tmp1 /tmp/mnt/tmp2; cp ~/fsstress
> /tmp/mnt
> ./fsstress -c -p 5 -n 10000 -d . -l 0 -s 10M &
> ./fsstress -c -p 5 -n 10000 -d ./tmp1 -l 0 -s 10M &
> ./fsstress -c -p 5 -n 10000 -d ./tmp2 -l 0 -s 10M &
>
> 11.when the following appears, run "ps;killall fsstress;ps"
> ./tmp1: No such file or directory
> ./tmp2: No such file or directory
>
> [2]- Exit 1 ./fsstress -c -p 5 -n 10000 -d ./tmp1 -l
> 0 -s 10M
> [3]+ Exit 1 ./fsstress -c -p 5 -n 10000 -d ./tmp2 -l
> 0 -s 10M
>
> These messages will appear:
>
> ps;killall fsstress;ps
> PID TTY TIME CMD
> 1366 pts/0 00:00:01 bash
> 1600 pts/0 00:00:00 fsstress
> 1633 pts/0 00:00:07 fsstress
> 1634 pts/0 00:00:06 fsstress
> 1635 pts/0 00:00:06 fsstress
> 1636 pts/0 00:00:07 fsstress
> 1637 pts/0 00:00:07 fsstress
> 1640 pts/0 00:00:00 ps
> [1]+ Terminated ./fsstress -c -p 5 -n 10000 -d . -l 0 -s
> 10M
> PID TTY TIME CMD
> 1366 pts/0 00:00:01 bash
> 1642 pts/0 00:00:00 ps
>
> 12. /bin/sync;/bin/echo 1 > /proc/sys/vm/drop_caches
>
> 13. rm -Rf p*;rm -Rf tmp*
>
> rm: cannot remove `p0': Directory not empty
> rm: cannot remove `p1/d99d/d9b4/da40/da71/da74': Directory not empty
> rm: cannot remove `p2/dc/d79/f56f': Input/output error
> rm: cannot remove `p2/dc/d79/f362': Input/output error
> rm: cannot remove `p2/dc/d79/c460': Input/output error
> rm: cannot remove `p2/dc/d79/cf4': Input/output error
> rm: cannot remove `p2/dc/d79/f3b0': Input/output error
> rm: cannot remove `p2/dc/d79/d107/d95e/d7f7/fb50': Input/output error
> rm: cannot remove `p2/dc/d79/d107/d95e/f99e': Input/output error
> rm: cannot remove `p2/dc/d79/d107/d95e/fc40': Input/output error
> rm: cannot remove `p2/dc/d79/d107/f7ab': Input/output error
> rm: cannot remove `p2/dc/d79/d107/d547/c843': Input/output error
> rm: cannot remove `p2/dc/d79/d107/d547/c871': Input/output error
> rm: cannot remove `p2/dc/d79/d107/fa77': Input/output error
> rm: cannot remove `p2/dc/d79/f12e': Input/output error
> rm: cannot remove `p2/dc/d79/c623': Input/output error
> rm: cannot remove `p2/dc/d79/f207': Input/output error
> rm: cannot remove `p2/dc/d79/c216': Input/output error
> rm: cannot remove `p2/dc/d79/f23d': Input/output error
> rm: cannot remove `p2/dc/d79/c250': Input/output error
> rm: cannot remove `p2/dc/d79/f274': Input/output error
> rm: cannot remove `p2/dc/d79/f285': Input/output error
> rm: cannot remove `p2/dc/d79/f2af': Input/output error
> rm: cannot remove `p2/dc/d79/c2d1': Input/output error
> rm: cannot remove `p2/dc/d79/c2ee': Input/output error
> rm: cannot remove `p2/dc/d79/c35a': Input/output error
> .....
>
> 14. run "dmesg"
>
> [ 469.002632] EXT4-fs warning (device loop0): dx_probe:364:
> Unrecognised inode hash code 12
> [ 469.002647] EXT4-fs warning (device loop0): dx_probe:472: Corrupt
> dir inode 8423, running e2fsck is recommended.
> [ 469.018830] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 8926
> [ 469.019792] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 8816
> [ 469.019950] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 8570
> [ 469.020729] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 7814
> [ 469.021227] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 9259
> [ 469.048610] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #10178: comm rm: deleted inode referenced: 7791
> [ 469.049919] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #10177: comm rm: deleted inode referenced: 8100
> [ 469.053826] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #10177: comm rm: deleted inode referenced: 8822
> [ 469.060481] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8417: comm rm: deleted inode referenced: 8113
> [ 469.062336] EXT4-fs warning (device loop0): dx_probe:364:
> Unrecognised inode hash code 12
> [ 469.062349] EXT4-fs warning (device loop0): dx_probe:472: Corrupt
> dir inode 10163, running e2fsck is recommended.
> [ 469.065721] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #10163: comm rm: deleted inode referenced: 9027
> [ 469.066138] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #10163: comm rm: deleted inode referenced: 9090
> [ 469.067526] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8417: comm rm: deleted inode referenced: 8443
> [ 469.068584] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 8067
> [ 469.069280] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 9694
> [ 469.069893] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 8100
> [ 469.070123] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 8635
> [ 469.070509] EXT4-fs error (device loop0): ext4_lookup:1044: inode
> #8423: comm rm: deleted inode referenced: 8929
> ......
>
> ***************************End**********************************************
>
>
> But if I use ext3, this will not occur.
> If I do not run "/bin/sync;/bin/echo 1 > /proc/sys/vm/drop_caches",
> this will not occur.
> If I use kernel>=3.0, this will not occur.
>
> But I use ext4 and run "/bin/sync; /bin/echo 1 >
> /proc/sys/vm/drop_caches", these error messages.
>
> So is this is a bug?
>
But if I use ext3, this will not occur.
If I do not run "/bin/sync;/bin/echo 1 > /proc/sys/vm/drop_caches", this
will not occur.
If I use kernel>=3.0, this will not occur.
If I use ext4 and run "/bin/sync; /bin/echo 1 >
/proc/sys/vm/drop_caches", these error messages will appear.
So is this is a bug?
On 03/12/2013 03:49 PM, zhuyj wrote:
> Hi, all
>
> Almost 2.6.3x has this problem. Is it a problem?
>
> Any reply is appreciated.
> Zhuyj
>
> On 03/12/2013 03:41 PM, zhuyj wrote:
>> Hi, all
>>
>> I install Ubuntu 11.04 on Dell-Latitude-D630.
>> Linux kernel is 2.6.39.4(I compiled)
>>
>> uname -a
>> Linux Dell-Latitude-D630 2.6.39.4-zhuyj26394 #2 SMP Tue Mar 12
>> 13:23:06 CST 2013 i686 i686 i386 GNU/Linux
>>
>> I followed these steps:
>>
>> ****************************Begin***************************************
>> 1. get ltp-full-20130109.bz2 and run "./configure;make", then we will
>> get fsstress;
>>
>> 2. mount -t tmpfs -o size=512m tmpfs /tmp
>>
>> 3. dd if=/dev/zero of=/tmp/img bs=1024 count=$((500*1024))
>>
>> 4. ls -lah /tmp/img
>>
>> total 501M
>> drwxrwxrwt 2 root root 60 2013-03-12 15:18 .
>> drwxr-xr-x 23 root root 4.0K 2013-03-12 13:09 ..
>> -rw-r--r-- 1 root root 500M 2013-03-12 15:18 img
>>
>> 5. df
>>
>> Filesystem 1K-blocks Used Available Use% Mounted on
>> /dev/sda1 113317544 6994416 100566868 7% /
>> none 1023772 696 1023076 1% /dev
>> none 1030440 80 1030360 1% /dev/shm
>> none 1030440 92 1030348 1% /var/run
>> none 1030440 0 1030440 0% /var/lock
>> tmpfs 524288 512504 11784 98% /tmp
>>
>> 6. mkfs.ext4 /tmp/img
>>
>> mke2fs 1.41.14 (22-Dec-2010)
>> /tmp/img is not a block special device.
>> Proceed anyway? (y,n) y
>> Filesystem label=
>> OS type: Linux
>> Block size=1024 (log=0)
>> Fragment size=1024 (log=0)
>> Stride=0 blocks, Stripe width=0 blocks
>> 128016 inodes, 512000 blocks
>> 25600 blocks (5.00%) reserved for the super user
>> First data block=1
>> Maximum filesystem blocks=67633152
>> 63 block groups
>> 8192 blocks per group, 8192 fragments per group
>> 2032 inodes per group
>> Superblock backups stored on blocks:
>> 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
>>
>> Writing inode tables: done
>> Creating journal (8192 blocks): done
>> Writing superblocks and filesystem accounting information: done
>>
>> This filesystem will be automatically checked every 30 mounts or
>> 180 days, whichever comes first. Use tune2fs -c or -i to override.
>>
>> 7. mkdir /tmp/mnt
>>
>> 8. mount -t ext4 -o loop /tmp/img /tmp/mnt
>>
>> 9. df
>> Filesystem 1K-blocks Used Available Use% Mounted on
>> /dev/sda1 113317544 6994424 100566860 7% /
>> none 1023772 696 1023076 1% /dev
>> none 1030440 80 1030360 1% /dev/shm
>> none 1030440 92 1030348 1% /var/run
>> none 1030440 0 1030440 0% /var/lock
>> tmpfs 524288 512504 11784 98% /tmp
>> /dev/loop0 495844 10510 459734 3% /tmp/mnt
>>
>> 10. cd /tmp/mnt; mkdir /tmp/mnt/tmp1 /tmp/mnt/tmp2; cp ~/fsstress
>> /tmp/mnt
>> ./fsstress -c -p 5 -n 10000 -d . -l 0 -s 10M &
>> ./fsstress -c -p 5 -n 10000 -d ./tmp1 -l 0 -s 10M &
>> ./fsstress -c -p 5 -n 10000 -d ./tmp2 -l 0 -s 10M &
>>
>> 11.when the following appears, run "ps;killall fsstress;ps"
>> ./tmp1: No such file or directory
>> ./tmp2: No such file or directory
>>
>> [2]- Exit 1 ./fsstress -c -p 5 -n 10000 -d ./tmp1
>> -l 0 -s 10M
>> [3]+ Exit 1 ./fsstress -c -p 5 -n 10000 -d ./tmp2
>> -l 0 -s 10M
>>
>> These messages will appear:
>>
>> ps;killall fsstress;ps
>> PID TTY TIME CMD
>> 1366 pts/0 00:00:01 bash
>> 1600 pts/0 00:00:00 fsstress
>> 1633 pts/0 00:00:07 fsstress
>> 1634 pts/0 00:00:06 fsstress
>> 1635 pts/0 00:00:06 fsstress
>> 1636 pts/0 00:00:07 fsstress
>> 1637 pts/0 00:00:07 fsstress
>> 1640 pts/0 00:00:00 ps
>> [1]+ Terminated ./fsstress -c -p 5 -n 10000 -d . -l 0
>> -s 10M
>> PID TTY TIME CMD
>> 1366 pts/0 00:00:01 bash
>> 1642 pts/0 00:00:00 ps
>>
>> 12. /bin/sync;/bin/echo 1 > /proc/sys/vm/drop_caches
>>
>> 13. rm -Rf p*;rm -Rf tmp*
>>
>> rm: cannot remove `p0': Directory not empty
>> rm: cannot remove `p1/d99d/d9b4/da40/da71/da74': Directory not empty
>> rm: cannot remove `p2/dc/d79/f56f': Input/output error
>> rm: cannot remove `p2/dc/d79/f362': Input/output error
>> rm: cannot remove `p2/dc/d79/c460': Input/output error
>> rm: cannot remove `p2/dc/d79/cf4': Input/output error
>> rm: cannot remove `p2/dc/d79/f3b0': Input/output error
>> rm: cannot remove `p2/dc/d79/d107/d95e/d7f7/fb50': Input/output error
>> rm: cannot remove `p2/dc/d79/d107/d95e/f99e': Input/output error
>> rm: cannot remove `p2/dc/d79/d107/d95e/fc40': Input/output error
>> rm: cannot remove `p2/dc/d79/d107/f7ab': Input/output error
>> rm: cannot remove `p2/dc/d79/d107/d547/c843': Input/output error
>> rm: cannot remove `p2/dc/d79/d107/d547/c871': Input/output error
>> rm: cannot remove `p2/dc/d79/d107/fa77': Input/output error
>> rm: cannot remove `p2/dc/d79/f12e': Input/output error
>> rm: cannot remove `p2/dc/d79/c623': Input/output error
>> rm: cannot remove `p2/dc/d79/f207': Input/output error
>> rm: cannot remove `p2/dc/d79/c216': Input/output error
>> rm: cannot remove `p2/dc/d79/f23d': Input/output error
>> rm: cannot remove `p2/dc/d79/c250': Input/output error
>> rm: cannot remove `p2/dc/d79/f274': Input/output error
>> rm: cannot remove `p2/dc/d79/f285': Input/output error
>> rm: cannot remove `p2/dc/d79/f2af': Input/output error
>> rm: cannot remove `p2/dc/d79/c2d1': Input/output error
>> rm: cannot remove `p2/dc/d79/c2ee': Input/output error
>> rm: cannot remove `p2/dc/d79/c35a': Input/output error
>> .....
>>
>> 14. run "dmesg"
>>
>> [ 469.002632] EXT4-fs warning (device loop0): dx_probe:364:
>> Unrecognised inode hash code 12
>> [ 469.002647] EXT4-fs warning (device loop0): dx_probe:472: Corrupt
>> dir inode 8423, running e2fsck is recommended.
>> [ 469.018830] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 8926
>> [ 469.019792] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 8816
>> [ 469.019950] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 8570
>> [ 469.020729] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 7814
>> [ 469.021227] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 9259
>> [ 469.048610] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #10178: comm rm: deleted inode referenced: 7791
>> [ 469.049919] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #10177: comm rm: deleted inode referenced: 8100
>> [ 469.053826] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #10177: comm rm: deleted inode referenced: 8822
>> [ 469.060481] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8417: comm rm: deleted inode referenced: 8113
>> [ 469.062336] EXT4-fs warning (device loop0): dx_probe:364:
>> Unrecognised inode hash code 12
>> [ 469.062349] EXT4-fs warning (device loop0): dx_probe:472: Corrupt
>> dir inode 10163, running e2fsck is recommended.
>> [ 469.065721] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #10163: comm rm: deleted inode referenced: 9027
>> [ 469.066138] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #10163: comm rm: deleted inode referenced: 9090
>> [ 469.067526] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8417: comm rm: deleted inode referenced: 8443
>> [ 469.068584] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 8067
>> [ 469.069280] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 9694
>> [ 469.069893] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 8100
>> [ 469.070123] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 8635
>> [ 469.070509] EXT4-fs error (device loop0): ext4_lookup:1044: inode
>> #8423: comm rm: deleted inode referenced: 8929
>> ......
>>
>> ***************************End**********************************************
>>
>>
>> But if I use ext3, this will not occur.
>> If I do not run "/bin/sync;/bin/echo 1 > /proc/sys/vm/drop_caches",
>> this will not occur.
>> If I use kernel>=3.0, this will not occur.
>>
>> But I use ext4 and run "/bin/sync; /bin/echo 1 >
>> /proc/sys/vm/drop_caches", these error messages.
>>
>> So is this is a bug?
>>
>
Hi Gao,
On Tue, Mar 12, 2013 at 02:58:24PM +0800, Lenky Gao wrote:
> Hi,
>
> When i just run a test on Centos 6.2 as follows:
> ################################
> #!/bin/bash
>
> while true
> do
>
> file="/tmp/filetest"
>
> echo $file
>
> dd if=/dev/zero of=${file} bs=512 count=204800 &> /dev/null
>
> sleep 5
> done
> ################################
>
> the inactive memory keep growing:
>
> #cat /proc/meminfo | grep Inactive\(fi
> Inactive(file): 420144 kB
> ...
> #cat /proc/meminfo | grep Inactive\(fi
> Inactive(file): 554662 kB
> ...
> #cat /proc/meminfo | grep Inactive\(fi
> Inactive(file): 882824 kB
>
> and i cannot reclaim it:
>
> # cat /proc/meminfo | grep Inactive\(file\);
> Inactive(file): 882824 kB
> # sync;
> # echo 3 > /proc/sys/vm/drop_caches
> # cat /proc/meminfo | grep Inactive\(file\);
> Inactive(file): 777664 kB
>
> My file system type is ext3, mounted with the option data=journal and
> it is easy to reproduce. If the mount option is data=ordered,
> reproducing this problom need more time. Because of the problom, when
> in the final situation, i cannot kmalloc a larger contiguous memory,
> especially in interrupt context. I have tested on other version
> kernel, such as 2.6.30 and 3.6.11, the problom also exists.
>
> Is this a bug?
I don't think it is a bug. If I understand correctly, when you try to
reclaim memory (sync and drop caches), the test script is still running,
right? If so, that means that some memory will be re-dirtied again. So
the number of inactive pages could not reduce as you expected.
Regards,
- Zheng
On Tue, Mar 12, 2013 at 4:37 PM, Zheng Liu <[email protected]> wrote:
> Hi Gao,
> I don't think it is a bug. If I understand correctly, when you try to
> reclaim memory (sync and drop caches), the test script is still running,
> right? If so, that means that some memory will be re-dirtied again. So
> the number of inactive pages could not reduce as you expected.
The test script has stopped.
Some log messages came from my UML Linux as follows:
...
/sdb/file_test
total used free shared buffers cached
Mem: 182 179 2 0 10 13
-/+ buffers/cache: 155 27
Swap: 0 0 0
/sdb/file_test
^C
[root@localhost ~]#
[root@localhost ~]# sync;
[root@localhost ~]# echo 3 > /proc/sys/vm/drop_caches
[root@localhost ~]#
[root@localhost ~]# echo 3 > /proc/sys/vm/drop_caches
[root@localhost ~]# sync;
[root@localhost ~]# echo 3 > /proc/sys/vm/drop_caches
[root@localhost ~]# free -m
total used free shared buffers cached
Mem: 182 157 25 0 0 3
-/+ buffers/cache: 153 28
Swap: 0 0 0
[root@localhost ~]# cat /proc/meminfo | grep Inac
Inactive: 124756 kB
Inactive(anon): 1408 kB
Inactive(file): 123348 kB
[root@localhost ~]# uname -a
Linux localhost.localdomain 2.6.30.8 #20 Mon Mar 11 07:04:06 EDT 2013
x86_64 x86_64 x86_64 GNU/Linux
[root@localhost ~]# mount | grep sdb
/dev/ubdb1 on /sdb type ext3 (rw,data=journal)
[root@localhost ~]# cat ./test_write_disk.sh
#!/bin/bash
while true
do
# file="/sdb/file_`date +%y%m%d%k%m%s`"
file="/sdb/file_test"
echo $file
dd if=/dev/zero of=${file} bs=512 count=20480 &> /dev/null
# sync
# cat /proc/meminfo | grep Inactive\(file\)
free -m
# rm -fr ${file}
done
--
Regards,
Lenky
On Tue, Mar 12, 2013 at 03:41:31PM +0800, zhuyj wrote:
> If I use kernel>=3.0, this will not occur.
Sounds like this is a problem in 2.6.39 that has since been fixed in
newer kernels.
> So is this is a bug?
Sure looks like it's a bug. :-)
There are so many distributions and other old embedded systems, etc.,
using older kernels that there's just no way that upstream developers
can try to debug every single older kernel --- and 2.6.39 isn't even a
kernel that is being supported by a volunteer as a long-term supported
kernel.
See http://www.kernel.org for a list of kernels which are supported as
long-term kernels, and even then, please remember that unless fixes
are automatically backported, or someone manually backports a fix that
doesn't automatically apply to an older kernel, it's not going to
happen.....
Regards,
- Ted
P.S. Your problem appears to be completely unrelated to the thread
which you replied to. This makes it hard for us to keep track of
questions/bug reports which users submit.
OK. Thanks a lot.
This bug can happen on Ubuntu 11.04,10.10 with kernel 2.6.3x.
Follow the above steps, this bug can be reproduced.
Anyone has the same experience?
On 03/12/2013 09:42 PM, Theodore Ts'o wrote:
> On Tue, Mar 12, 2013 at 03:41:31PM +0800, zhuyj wrote:
>> If I use kernel>=3.0, this will not occur.
> Sounds like this is a problem in 2.6.39 that has since been fixed in
> newer kernels.
>
>> So is this is a bug?
> Sure looks like it's a bug. :-)
>
> There are so many distributions and other old embedded systems, etc.,
> using older kernels that there's just no way that upstream developers
> can try to debug every single older kernel --- and 2.6.39 isn't even a
> kernel that is being supported by a volunteer as a long-term supported
> kernel.
>
> See http://www.kernel.org for a list of kernels which are supported as
> long-term kernels, and even then, please remember that unless fixes
> are automatically backported, or someone manually backports a fix that
> doesn't automatically apply to an older kernel, it's not going to
> happen.....
>
> Regards,
>
> - Ted
>
> P.S. Your problem appears to be completely unrelated to the thread
> which you replied to. This makes it hard for us to keep track of
> questions/bug reports which users submit.
>
Hi Everyone,
Maybe i have found the answer for this question. The author of the JBD
have explained in the comments:
/*
* When an ext3-ordered file is truncated, it is possible that many pages are
* not successfully freed, because they are attached to a committing
transaction.
* After the transaction commits, these pages are left on the LRU, with no
* ->mapping, and with attached buffers. These pages are trivially reclaimable
* by the VM, but their apparent absence upsets the VM accounting, and it makes
* the numbers in /proc/meminfo look odd.
...
*/
static void release_buffer_page(struct buffer_head *bh)
{
struct page *page;
...
But my new question is why not free those pages after the transaction commits?
--
Regards,
Lenky
Hi, all
The two patches can fix this problem.
ext4: Fix fs corruption when make_indexed_dir() fails
ext4: don't dereference null pointer when make_indexed_dir() fails
If any one comes across this problem, he can apply the above two patches.
Zhuyj
On 03/13/2013 11:24 AM, zhuyj wrote:
> OK. Thanks a lot.
> This bug can happen on Ubuntu 11.04,10.10 with kernel 2.6.3x.
> Follow the above steps, this bug can be reproduced.
>
> Anyone has the same experience?
>
> On 03/12/2013 09:42 PM, Theodore Ts'o wrote:
>> On Tue, Mar 12, 2013 at 03:41:31PM +0800, zhuyj wrote:
>>> If I use kernel>=3.0, this will not occur.
>> Sounds like this is a problem in 2.6.39 that has since been fixed in
>> newer kernels.
>>
>>> So is this is a bug?
>> Sure looks like it's a bug. :-)
>>
>> There are so many distributions and other old embedded systems, etc.,
>> using older kernels that there's just no way that upstream developers
>> can try to debug every single older kernel --- and 2.6.39 isn't even a
>> kernel that is being supported by a volunteer as a long-term supported
>> kernel.
>>
>> See http://www.kernel.org for a list of kernels which are supported as
>> long-term kernels, and even then, please remember that unless fixes
>> are automatically backported, or someone manually backports a fix that
>> doesn't automatically apply to an older kernel, it's not going to
>> happen.....
>>
>> Regards,
>>
>> - Ted
>>
>> P.S. Your problem appears to be completely unrelated to the thread
>> which you replied to. This makes it hard for us to keep track of
>> questions/bug reports which users submit.
>>
>
>
On Fri, Mar 15, 2013 at 03:16:52PM +0800, zhuyj wrote:
> Hi, all
>
> The two patches can fix this problem.
> ext4: Fix fs corruption when make_indexed_dir() fails
> ext4: don't dereference null pointer when make_indexed_dir() fails
Thanks, Zhujh for figuring this out. Those two patches weren't
taggled with cc: [email protected], but they probably should have
been. Since they first appeared in v3.0, the two long-term stable
kernels that would benefit with these patches are v3.0.x and v3.2.x.
Willy, Paul, would you consider cherry-picking the following commits
for your respective stable kernels?
git cherry-pick 7ad8e4e6ae2a7c95445ee1715b1714106fb95037
git cherry-pick 6976a6f2acde2b0443cd64f1d08af90630e4ce81
Thanks!!
- Ted
On Fri, Mar 15, 2013 at 10:53:14AM -0400, Theodore Ts'o wrote:
> On Fri, Mar 15, 2013 at 03:16:52PM +0800, zhuyj wrote:
> > Hi, all
> >
> > The two patches can fix this problem.
> > ext4: Fix fs corruption when make_indexed_dir() fails
> > ext4: don't dereference null pointer when make_indexed_dir() fails
>
> Thanks, Zhujh for figuring this out. Those two patches weren't
> taggled with cc: [email protected], but they probably should have
> been. Since they first appeared in v3.0, the two long-term stable
> kernels that would benefit with these patches are v3.0.x and v3.2.x.
>
> Willy, Paul, would you consider cherry-picking the following commits
> for your respective stable kernels?
>
> git cherry-pick 7ad8e4e6ae2a7c95445ee1715b1714106fb95037
> git cherry-pick 6976a6f2acde2b0443cd64f1d08af90630e4ce81
Yes of course !
Thank you Ted!
Willy
On Fri, Mar 15, 2013 at 03:56:04PM +0100, Willy Tarreau wrote:
> > Thanks, Zhujh for figuring this out. Those two patches weren't
> > taggled with cc: [email protected], but they probably should have
> > been. Since they first appeared in v3.0, the two long-term stable
> > kernels that would benefit with these patches are v3.0.x and v3.2.x.
Oops, sorry, that should have read 2.6.32 and 2.6.34. :-)
> > Willy, Paul, would you consider cherry-picking the following commits
> > for your respective stable kernels?
> >
> > git cherry-pick 7ad8e4e6ae2a7c95445ee1715b1714106fb95037
> > git cherry-pick 6976a6f2acde2b0443cd64f1d08af90630e4ce81
>
> Yes of course !
Great, thanks!!
- Ted
On 13-03-15 10:53 AM, Theodore Ts'o wrote:
> On Fri, Mar 15, 2013 at 03:16:52PM +0800, zhuyj wrote:
>> Hi, all
>>
>> The two patches can fix this problem.
>> ext4: Fix fs corruption when make_indexed_dir() fails
>> ext4: don't dereference null pointer when make_indexed_dir() fails
>
> Thanks, Zhujh for figuring this out. Those two patches weren't
> taggled with cc: [email protected], but they probably should have
> been. Since they first appeared in v3.0, the two long-term stable
> kernels that would benefit with these patches are v3.0.x and v3.2.x.
>
> Willy, Paul, would you consider cherry-picking the following commits
> for your respective stable kernels?
Both queued for 2.6.34.15
Thanks,
Paul.
--
>
> git cherry-pick 7ad8e4e6ae2a7c95445ee1715b1714106fb95037
> git cherry-pick 6976a6f2acde2b0443cd64f1d08af90630e4ce81
>
> Thanks!!
>
> - Ted
>