Is this feature or bug?
First a test on 'stable' 2.2.x kernel:
eax /tmp % uname -a
Linux eax 2.2.17pre15 #1 Sat Aug 5 14:31:19 EDT 2000 i586 unknown
eax /tmp % dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
1000+0 records in
1000+0 records out
eax /tmp % ls -l holed.file
-rw-rw-r-- 1 adam adam 6000000 Nov 29 08:57 holed.file
eax /tmp % du -sh holed.file
983k holed.file
Above holey file is as expected aproximately 1mb.
However, when I try on 'development' 2.4.x kernel
[adam@pepsi /tmp]$ uname -a
Linux pepsi 2.4.0-test7-packet #24 SMP Fri Sep 8 20:26:35 EDT 2000 i686
[adam@pepsi /tmp]$ dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
1000+0 records in
1000+0 records out
[adam@pepsi /tmp]$ ls -l holed.file
-rw-rw-r-- 1 adam adam 6000000 Nov 29 08:52 holed.file
[adam@pepsi /tmp]$ du -sh holed.file
1.9M holed.file
The holey file is twice as big,
at almost 2mb instead of expected 1mb.
--
Adam
http://www.eax.com The Supreme Headquarters of the 32 bit registers
On Wed, 29 Nov 2000, Adam wrote:
> However, when I try on 'development' 2.4.x kernel
>
> [adam@pepsi /tmp]$ uname -a
> Linux pepsi 2.4.0-test7-packet #24 SMP Fri Sep 8 20:26:35 EDT 2000 i686
>
> [adam@pepsi /tmp]$ dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
> 1000+0 records in
> 1000+0 records out
>
> [adam@pepsi /tmp]$ ls -l holed.file
> -rw-rw-r-- 1 adam adam 6000000 Nov 29 08:52 holed.file
>
> [adam@pepsi /tmp]$ du -sh holed.file
> 1.9M holed.file
>
what filesystem type? on ext2 filesystem on 2.4.0-test12-pre3 I get
expected result:
# dd if=/dev/zero of=hole bs=1000 seek=5000 count=1000
1000+0 records in
1000+0 records out
# l hole
-rw-r--r-- 1 root root 6000000 Nov 29 13:06 hole
# du -sh hole
988k hole
# u
Linux penguin 2.4.0-test12 #1 Wed Nov 29 09:08:13 GMT 2000 i686 unknown
Regards,
Tigran
> > [adam@pepsi /tmp]$ uname -a
> > Linux pepsi 2.4.0-test7-packet #24 SMP Fri Sep 8 20:26:35 EDT 2000 i686
> >
> > [adam@pepsi /tmp]$ dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
> > [adam@pepsi /tmp]$ ls -l holed.file
> > -rw-rw-r-- 1 adam adam 6000000 Nov 29 08:52 holed.file
> > [adam@pepsi /tmp]$ du -sh holed.file
> > 1.9M holed.file
> what filesystem type? on ext2 filesystem on 2.4.0-test12-pre3 I get
> expected result:
>
> # dd if=/dev/zero of=hole bs=1000 seek=5000 count=1000
> 1000+0 records in
> 1000+0 records out
> # l hole
> -rw-r--r-- 1 root root 6000000 Nov 29 13:06 hole
> # du -sh hole
> 988k hole
> # u
> Linux penguin 2.4.0-test12 #1 Wed Nov 29 09:08:13 GMT 2000 i686 unknown
It is ext2fs with 4kb blocks.
[root@pepsi /tmp]# rpm -q -f /usr/bin/du
fileutils-4.0-21
[root@pepsi /tmp]# ldd /usr/bin/du
libc.so.6 => /lib/libc.so.6 (0x4001f000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
[root@pepsi /tmp]# rpm -q -f /lib/libc.so.6
glibc-2.1.3-21
--
Adam
http://www.eax.com The Supreme Headquarters of the 32 bit registers
On Wed, 29 Nov 2000, Tigran Aivazian wrote:
> > [adam@pepsi /tmp]$ dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
> > [adam@pepsi /tmp]$ ls -l holed.file
> > -rw-rw-r-- 1 adam adam 6000000 Nov 29 08:52 holed.file
> > [adam@pepsi /tmp]$ du -sh holed.file
> > 1.9M holed.file
>
> what filesystem type? on ext2 filesystem on 2.4.0-test12-pre3 I get
> expected result:
More datapoints. I have asked around, and I have two users of
2.4.0-test10. One is getting expected 1mb, other is getting (just like me)
1.9mb.
So far I don't see pattern, here. It does not seems to depend on block
size, nor packet writing patches.
--
Adam
http://www.eax.com The Supreme Headquarters of the 32 bit registers
On Wed, 29 Nov 2000, Adam wrote:
> On Wed, 29 Nov 2000, Tigran Aivazian wrote:
>
> > > [adam@pepsi /tmp]$ dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
> > > [adam@pepsi /tmp]$ ls -l holed.file
> > > -rw-rw-r-- 1 adam adam 6000000 Nov 29 08:52 holed.file
> > > [adam@pepsi /tmp]$ du -sh holed.file
> > > 1.9M holed.file
> >
> > what filesystem type? on ext2 filesystem on 2.4.0-test12-pre3 I get
> > expected result:
>
> More datapoints. I have asked around, and I have two users of
> 2.4.0-test10. One is getting expected 1mb, other is getting (just like me)
> 1.9mb.
>
> So far I don't see pattern, here. It does not seems to depend on block
> size, nor packet writing patches.
I don't use ext2, I use reiserfs and it works fine here (test10)
tux:~$dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
1000+0 records in
1000+0 records out
tux:~$ls -l holed.file
-rw-r--r-- 1 gandalf gandalf 6000000 Nov 29 20:55 holed.file
tux:~$du -sh holed.file
980k holed.file
tested on a router (with test10 and ext2 and reiserfs) and it works fine
on both filsystems
tested on another router with reiserfs and ext2 running test11
works fine there too on both filesystems
works fine on test8 with both ext2 and reiserfs
(1k blocksize on ext2 and 4k blocksize on reiserfs)
/Martin
kernel 2.2.17, '/' being a 1k blocksize ext2fs:
root@adam:/ > dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
1000+0 records in
1000+0 records out
root@adam:/ > ls -l holed.file
-rw-r--r-- 1 root root 6000000 Nov 29 23:33 holed.file
root@adam:/ > du -sh holed.file
5.7M holed.file
Now that seems funny.
Marc
--
Marc Mutz <[email protected]> http://EncryptionHOWTO.sourceforge.net/
University of Bielefeld, Dep. of Mathematics / Dep. of Physics
PGP-keyID's: 0xd46ce9ab (RSA), 0x7ae55b9e (DSS/DH)
On Wed, 29 Nov 2000 17:36:12 Marc Mutz wrote:
> kernel 2.2.17, '/' being a 1k blocksize ext2fs:
>
> root@adam:/ > dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
> 1000+0 records in
> 1000+0 records out
> root@adam:/ > ls -l holed.file
> -rw-r--r-- 1 root root 6000000 Nov 29 23:33 holed.file
> root@adam:/ > du -sh holed.file
> 5.7M holed.file
>
> Now that seems funny.
why is 5.7MB funny with 6000000 bytes ? or are you talking about
something else? Don't forget that BYTES and thus MEGABYTES are found by
powers of 2. They do not function like the decimal "bits". quick way to
do the conversion is divide bytes by 1024 to get Kilobytes and then again
by 1024 to get Megabytes. This is how it should be displayed. Or am i
missing the point of your comment?
> Marc
>
> --
> Marc Mutz <[email protected]> http://EncryptionHOWTO.sourceforge.net/
> University of Bielefeld, Dep. of Mathematics / Dep. of Physics
>
> PGP-keyID's: 0xd46ce9ab (RSA), 0x7ae55b9e (DSS/DH)
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in
> the body of a message to [email protected]
> Please read the FAQ at http://www.tux.org/lkml/
>
Marc Mutz writes:
> kernel 2.2.17, '/' being a 1k blocksize ext2fs:
>
> root@adam:/ > dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
> 1000+0 records in
> 1000+0 records out
> root@adam:/ > ls -l holed.file
> -rw-r--r-- 1 root root 6000000 Nov 29 23:33 holed.file
> root@adam:/ > du -sh holed.file
> 5.7M holed.file
Strangely, I have 2.2.17 (TurboLinux patched), on a 1k filesystem and
I have no problems. I have 1k, 2k, and 4k ext2 fs, all OK.
What people who have the problem should be doing is:
> ls -li holed.file # find inode number
10732 -rw-r--r-- 1 root root 6000000 Nov 29 16:17 holed.file
> du -sk holed.file # see what "stat" thinks
983k holed.file
> debugfs /dev/XXX
debugfs> stats # find out ext2 block size
...
Block size = 1024, fragment size = 1024
...
debugfs> stat <10732> # (with < and >)
Inode: 10732 Type: regular Mode: 0644 Flags: 0x0 Generation:
4048594821
User: 0 Group: 0 Size: 6000000
File ACL: 0 Directory ACL: 0
Links: 1 Blockcount: 1966
^^^^ these are 512-byte blocks, so / 2 for ~kB
they include indirect blocks and such
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x3a258e82 -- Wed Nov 29 16:17:22 2000
atime: 0x3a258e82 -- Wed Nov 29 16:17:22 2000
mtime: 0x3a258e82 -- Wed Nov 29 16:17:22 2000
BLOCKS:
47512 47513 47514 47515 47516 47517 47518 47519 47520 ... 48723 48724
TOTAL: 983
^^^ these are ext2fs sized blocks, not necessarily kB
If what debugfs says doesn't match du, then it is du/libc/stat that is
broken. If debugfs says the file actually has 6000000 bytes of data,
then it is the filesystem that is broken.
Cheers, Andreas
--
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert
On Wed, 29 Nov 2000, Andreas Dilger wrote:
> What people who have the problem should be doing is:
> [desc snipped]
> > ls -li holed.file # find inode number
> 10732 -rw-r--r-- 1 root root 6000000 Nov 29 16:17 holed.file
> > du -sk holed.file # see what "stat" thinks
> 983k holed.file
> > debugfs /dev/XXX
> debugfs> stats # find out ext2 block size
> ...
> Block size = 1024, fragment size = 1024
> ...
> debugfs> stat <10732> # (with < and >)
> Inode: 10732 Type: regular Mode: 0644 Flags: 0x0 Generation:
> 4048594821
> User: 0 Group: 0 Size: 6000000
> File ACL: 0 Directory ACL: 0
> Links: 1 Blockcount: 1966
> ^^^^ these are 512-byte blocks, so / 2 for ~kB
> they include indirect blocks and such
> Fragment: Address: 0 Number: 0 Size: 0
> ctime: 0x3a258e82 -- Wed Nov 29 16:17:22 2000
> atime: 0x3a258e82 -- Wed Nov 29 16:17:22 2000
> mtime: 0x3a258e82 -- Wed Nov 29 16:17:22 2000
> BLOCKS:
> 47512 47513 47514 47515 47516 47517 47518 47519 47520 ... 48723 48724
> TOTAL: 983
> ^^^ these are ext2fs sized blocks, not necessarily kB
>
> If what debugfs says doesn't match du, then it is du/libc/stat that is
> broken. If debugfs says the file actually has 6000000 bytes of data,
> then it is the filesystem that is broken.
I just did what suggested, and it seems that DU reports correct values,
I have attached 'sript' log of the above example on my filesystem.
Here are some highlights:
[adam@pepsi /tmp]$ ls -lis holed.file
3085069 5872 -rw-rw-r-- 1 adam adam 6000000 Nov 30 09:11 holed.file
Block size = 4096, fragment size = 4096
Links: 1 Blockcount: 11744
TOTAL: 1468
so it seems DU reports correct values as :
1966/2 =983
11744/2=5872
and
4096*1468=6012928
1024*983 =1006592
--
Adam
http://www.eax.com The Supreme Headquarters of the 32 bit registers
Adam writes:
> I just did what suggested, and it seems that DU reports correct values,
> I have attached 'sript' log of the above example on my filesystem.
> Here are some highlights:
>
> [adam@pepsi /tmp]$ ls -lis holed.file
> 3085069 5872 -rw-rw-r-- 1 adam adam 6000000 Nov 30 09:11 holed.file
>
> Block size = 4096, fragment size = 4096
> Links: 1 Blockcount: 11744
> TOTAL: 1468
>
> so it seems DU reports correct values as :
> 11744/2=5872
> and
> 4096*1468=6012928
I guess the next thing to check is if your "dd" is actually seeking, or
just pretending to... Maybe an strace of the "dd" call will tell us if
it is screwing with our minds. Also, if you could make a scratch ext2
filesystem with 1k blocks, and see if it does the same thing. Even
better would be to try a different kernel to see if it affects this.
Cheers, Andreas
--
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert
Andreas Dilger wrote:
>
> Marc Mutz writes:
> > kernel 2.2.17, '/' being a 1k blocksize ext2fs:
> >
> > root@adam:/ > dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
> > 1000+0 records in
> > 1000+0 records out
> > root@adam:/ > ls -l holed.file
> > -rw-r--r-- 1 root root 6000000 Nov 29 23:33 holed.file
> > root@adam:/ > du -sh holed.file
> > 5.7M holed.file
>
> Strangely, I have 2.2.17 (TurboLinux patched), on a 1k filesystem and
> I have no problems. I have 1k, 2k, and 4k ext2 fs, all OK.
>
> What people who have the problem should be doing is:
> > ls -li holed.file # find inode number
> 10732 -rw-r--r-- 1 root root 6000000 Nov 29 16:17 holed.file
> > du -sk holed.file # see what "stat" thinks
> 983k holed.file
dito
> > debugfs /dev/XXX
> debugfs> stats # find out ext2 block size
> ...
> Block size = 1024, fragment size = 1024
> ...
dito
> Links: 1 Blockcount: 1966
dito
> TOTAL: 983
dito
> If what debugfs says doesn't match du, then it is du/libc/stat that is
> broken. If debugfs says the file actually has 6000000 bytes of data,
> then it is the filesystem that is broken.
>
$ du --version
du (GNU fileutils) 3.16
$ ls -l /lib/libc*
lrwxrwxrwx 1 root root 13 May 8 1999 /lib/libc.so.4 \
-> libc.so.4.7.6
-rwxr-xr-x 1 root root 634880 Apr 29 1996
/lib/libc.so.4.7.6
-rwxr-xr-x 1 root root 2478585 Dec 14 1998 /lib/libc.so.6
-rwxr-xr-x 1 root root 85699 Dec 14 1998
/lib/libcrypt.so.1
$ ldd /usr/bin/du
libc.so.6 => /lib/libc.so.6 (0x40009000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
all from SuSE 6.0 (I know, it's old...)
Marc
--
Marc Mutz <[email protected]> http://EncryptionHOWTO.sourceforge.net/
University of Bielefeld, Dep. of Mathematics / Dep. of Physics
PGP-keyID's: 0xd46ce9ab (RSA), 0x7ae55b9e (DSS/DH)
Hi!
On Sat, 2 Dec 2000, Adam wrote:
> It seems you are right. If I remove the file first, then it will show
> correct amount.
(For the list: In private mail it was discovered that this behaviour was
caused by the file 'holed.file' not being empty before the dd command.)
dd behaves here correctly. It does not append to file, but it just
opens the file for writing. It does not remove or truncate it first.
If you have executed this command:
dd if=/dev/zero of=holed.file bs=1000 count=1000
You have a simple file with zeroes, like this:
'000000000'
Then with this command dd opens the file for write but does not destroy
its contents - it is not supposed to do that. (You can alter files with
dd, you can overwrite single bytes if you like.)
dd if=/dev/zero of=holed.file bs=1000 seek=5000 count=1000
After opening the file dd seeks to what you specify, and at the same time
extends the file by seeking and Linux converts this to holes:
'000000000-----------------------------------------'
And after seeking it writes the new block of zeroes:
'000000000-----------------------------------------000000000'
..and you have 2MB of stuff there instead of 1M!
Yours,
Jaakko
--
Weather Service Finland Ltd [email protected]
Pursimiehenkatu 29-31 B, FIN-00150 Helsinki, Finland http://www.weather.fi