I've found some undesirable behavior when trying to use the loop block
driver with files with strange byte counts. The files I'd like to setup
as loopback devices are on read-only media, and they're all different byte
counts... not necessarily even an integral multiple of 512. I can't just
pad them to work with losetup.
Here's the scoop on strange filesizes.
Sometimes, the loopback driver takes a file, and calculates the greatest
number of blocks that will fit in the file, and uses that. It
understandably chops off the remaining bytes that don't quite add up to a
full block. The amount of data you get when reading the device and the
return from BLKGETSIZE are the same. This is good.
Other times, the loopback driver will do something kinda funky... it will
once again figure out block counts that will fit in the file, but it
reports one thing in BLKGETSIZE, but provides less data than you actually
get when you try to read (dd, md5sum, whatever) the /dev/loopN device.
This is bad. I get an I/O error when I try to read such a loopback
device.
Which of the two above paths is taken seems to be a rounding issue, as far
as I can tell.
If you have a 1048576-byte file (exactly 1M), losetup works perfectly.
You get every byte. BLKGETSIZE says 2048.
If you have a 1048577-byte file (1 byte more than 1M), the loopback device
skips the last byte, which is what I believe it should do. BLKGETSIZE
says 2048.
If you have a 1048575-byte file (1 byte less than 1M), the loopback device
reports 2046 for it's BLKGETSIZE. This sounds correct, since it has to
use one less 1024-block to get an integral multiple. However, when you
examine the contents of the loopback device, you'll find that there are
only 2040 sectors... there are 3072 more bytes missing than you'd expect.
So what is the block size we're dealing with here? 1024 or 4096? I don't
really care how much gets chopped off the end of the loopback device...
just so long as the results are predictable, and that BLKGETSIZE matches
the actual amount of data that can be read from the loopback device.
I'd love to help solve this problem by myself... in fact I've spent the
last several hours trying. I think I need the aid of a more experienced
kernel hacker, though. :)
BTW...
I'm not a subscriber to this list, so I'd greatly appreciate any replies
being cc'ed to me.
Thanks!
-Jeff Meininger
From: Jeff Meininger <[email protected]>
I've found some undesirable behavior when trying to use the loop block
driver with files with strange byte counts. The files I'd like to setup
as loopback devices are on read-only media, and they're all different byte
counts... not necessarily even an integral multiple of 512. I can't just
pad them to work with losetup.
What kernel version?
If you have a 1048577-byte file (1 byte more than 1M), the loopback device
skips the last byte, which is what I believe it should do. BLKGETSIZE
says 2048.
If you have a 1048575-byte file (1 byte less than 1M), the loopback device
reports 2046 for it's BLKGETSIZE. This sounds correct, since it has to
use one less 1024-block to get an integral multiple. However, when you
examine the contents of the loopback device, you'll find that there are
only 2040 sectors... there are 3072 more bytes missing than you'd expect.
No doubt what happens is that the underlying filesystem
that contains the file that you do losetup on has a
block size 4096.
You can check by asking
# blockdev --getbsz /dev/loopN
If the answer is 4096, then you can change it by
# blockdev --setbsz 512 /dev/loopN
With the wrong block size, you will get an I/O error
reading the end of the loop device. With the right
block size there will not be an I/O error, but of course
still the size is rounded down to a multiple of 1024.
There is a growing need to address Linux' inability to
get at the last sector of devices with an odd number of
sectors, and more generally to get at the end of block devices
with a length that is not a nice multiple of a large power of 2.
Andries
Jeff Meininger wrote:
> I've found some undesirable behavior when trying to use the loop block
> driver with files with strange byte counts. The files I'd like to setup
> as loopback devices are on read-only media, and they're all different byte
> counts... not necessarily even an integral multiple of 512. I can't just
> pad them to work with losetup.
[snip]
> I'd love to help solve this problem by myself... in fact I've spent the
> last several hours trying. I think I need the aid of a more experienced
> kernel hacker, though. :)
This bug has been around since 2.4.6pre days and it is still unfixed in
mainline kernels. I have posted patch fix this bug twice, once in 2.4.6pre
days and once in 2.4.17pre days. My patches were ignored as usual.
Just for the record, this bug has been fixed in loop-AES since version v1.3b
(June 27 2001). Latest version is here:
http://loop-aes.sourceforge.net/loop-AES-v1.5b.tar.bz2
Regards,
Jari Ruusu <[email protected]>