Hi all,
I ran a small test which creates one directory and 2O 8-KB size files in it.
When the filesystem is mounted without the delalloc option, here is the
output of the command dumpe2fs for the group in which the directory and
the files are created:
Group 532 : (Blocks 17432576-17465343)
Block bitmap at 17432576 (+0), Inode bitmap at 17432577 (+1)
Inode table at 17432578-17433089 (+2)
32213 free blocks, 16363 free inodes, 1 directories
Free blocks : 17433090-17459199, 17459241-17465343
Free inodes : 8716310-8732672
When the filesystem is mounted with the delalloc option, the same test
gives a different result:
Group 395 : (Blocks 12943360-12976127)
Block bitmap at 12943360 (+0), Inode bitmap at 12943361 (+1)
Inode table at 12943362-12943873 (+2)
32213 free blocks, 16363 free inodes, 1 directories
Free blocks : 12943874-12955647, 12955650-12955655,
12955658-12955663, 12955666-12955671, 12955674-12955679,
12955682-12955687, 12955690-12955695, 12955698-12955703,
12955706-12955711, 12955714-12955719, 12955722-12955727,
12955730-12955735, 12955738-12955743, 12955746-12955751,
12955754-12955759, 12955762-12955767, 12955770-12955775,
12955778-12955783, 12955786-12955791, 12955794-12955799,
12955802-12961791, 12961793-12976127
Free inodes : 6471702-6488064
In the first case, the allocated blocks are contiguous whereas they are
not in the second case.
After adding traces in the code to understand why the behavior is
different with the delalloc option, I found that the problem is related
to the inode reservation window.
To simplify, without the delalloc option we have the following scenario:
For each inode,
- call alloc_new_reservation() to allocate a new reservation window
- allocate blocks for data
- write data to disk
- ext4_discard_reservation() when the inode is closed.
With the delalloc option, when the data are written to disk we have:
For each inode,
- call alloc_new_reservation() to allocate a new reservation window
- allocate blocks for data
- write data to disk
I think a call to ext4_discard_reservation() is missing somewhere and
the question is where.
I tried to add this call at the end of the ext4_da_get_block_write()
function. This seems to fix the problem as the blocks are allocated
contiguously on disk but the function seems to be called too many times
so I think it is perhaps not the right place to call it.
Who could look into this problem?
I've got a few days off so I couldn't help more next days, but the
problem is easily reproductible.
Wouldn't this also explain why the compilebench results posted by Chris
Mason are bad in some cases?
Val?rie
Valerie Clement wrote:
> Hi all,
>
> I ran a small test which creates one directory and 2O 8-KB size files in
> it.
>
> When the filesystem is mounted without the delalloc option, here is the
> output of the command dumpe2fs for the group in which the directory and
> the files are created:
>
> Group 532 : (Blocks 17432576-17465343)
> Block bitmap at 17432576 (+0), Inode bitmap at 17432577 (+1)
> Inode table at 17432578-17433089 (+2)
> 32213 free blocks, 16363 free inodes, 1 directories
> Free blocks : 17433090-17459199, 17459241-17465343
> Free inodes : 8716310-8732672
>
>
> When the filesystem is mounted with the delalloc option, the same test
> gives a different result:
>
> Group 395 : (Blocks 12943360-12976127)
> Block bitmap at 12943360 (+0), Inode bitmap at 12943361 (+1)
> Inode table at 12943362-12943873 (+2)
> 32213 free blocks, 16363 free inodes, 1 directories
> Free blocks : 12943874-12955647, 12955650-12955655, 12955658-12955663,
> 12955666-12955671, 12955674-12955679, 12955682-12955687,
> 12955690-12955695, 12955698-12955703, 12955706-12955711,
> 12955714-12955719, 12955722-12955727, 12955730-12955735,
> 12955738-12955743, 12955746-12955751, 12955754-12955759,
> 12955762-12955767, 12955770-12955775, 12955778-12955783,
> 12955786-12955791, 12955794-12955799, 12955802-12961791, 12961793-12976127
> Free inodes : 6471702-6488064
>
I am finding both the case to be equally distributed. Tried
-o mballoc,delalloc and -o nomballoc,nodelalloc and ext3
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20; do dd if=/dev/zero of=mytest$i bs=1024 count=8; done
nodelalloc,nomballoc
---------------
Free blocks: 25017-25088, 25097-25600, 25609-26112, 26121-26624, 26633-27136, 27145-27648, 27657-28160, 28169-28672, 28682-29184, 29201-29696, 29713-30208, 30225-30720, 30737-31232, 31241-31744, 31753-32256, 32265-32768
delalloc,mballoc
-----------
Free blocks: 16624-16896, 16905-17408, 17417-17920, 17929-18432, 18441-18944, 18954-19456, 19465-19968, 19985-20480, 20497-20992, 21009-21504, 21521-22016, 22025-22528, 22537-23040, 23049-23552, 23561-24064, 24073-24576
with ext3
--------
Free blocks: 25017-25088, 25097-25600, 25609-26112, 26121-26624, 26633-27136, 27145-27648, 27658-28160, 28169-28672, 28689-29184, 29201-29696, 29713-30208, 30225-30720, 30729-31232, 31241-31744, 31753-32256, 32265-32768
On Fri, 2007-10-26 at 14:28 +0200, Valerie Clement wrote:
> Hi all,
>
Hi Valerie,
> I ran a small test which creates one directory and 2O 8-KB size files in it.
>
> When the filesystem is mounted without the delalloc option, here is the
> output of the command dumpe2fs for the group in which the directory and
> the files are created:
>
> Group 532 : (Blocks 17432576-17465343)
> Block bitmap at 17432576 (+0), Inode bitmap at 17432577 (+1)
> Inode table at 17432578-17433089 (+2)
> 32213 free blocks, 16363 free inodes, 1 directories
> Free blocks : 17433090-17459199, 17459241-17465343
> Free inodes : 8716310-8732672
>
>
> When the filesystem is mounted with the delalloc option, the same test
> gives a different result:
>
> Group 395 : (Blocks 12943360-12976127)
> Block bitmap at 12943360 (+0), Inode bitmap at 12943361 (+1)
> Inode table at 12943362-12943873 (+2)
> 32213 free blocks, 16363 free inodes, 1 directories
> Free blocks : 12943874-12955647, 12955650-12955655,
> 12955658-12955663, 12955666-12955671, 12955674-12955679,
> 12955682-12955687, 12955690-12955695, 12955698-12955703,
> 12955706-12955711, 12955714-12955719, 12955722-12955727,
> 12955730-12955735, 12955738-12955743, 12955746-12955751,
> 12955754-12955759, 12955762-12955767, 12955770-12955775,
> 12955778-12955783, 12955786-12955791, 12955794-12955799,
> 12955802-12961791, 12961793-12976127
> Free inodes : 6471702-6488064
>
> In the first case, the allocated blocks are contiguous whereas they are
> not in the second case.
>
> After adding traces in the code to understand why the behavior is
> different with the delalloc option, I found that the problem is related
> to the inode reservation window.
> To simplify, without the delalloc option we have the following scenario:
> For each inode,
> - call alloc_new_reservation() to allocate a new reservation window
> - allocate blocks for data
> - write data to disk
> - ext4_discard_reservation() when the inode is closed.
>
> With the delalloc option, when the data are written to disk we have:
> For each inode,
> - call alloc_new_reservation() to allocate a new reservation window
> - allocate blocks for data
> - write data to disk
>
>
> I think a call to ext4_discard_reservation() is missing somewhere and
> the question is where.
>
Oh, that should be block reservation, not inode reservation window.
The problem with delayed allocation and block reservation is, we don't
know when suppose to close the window, as the file maybe closed with
diry data in cache,and the blocks has not be allocated yet. We would
like to keep the window open so that later delayed allocation happens,
the allocation could take advantage of the reservation. But on the other
hand, that may leads fs external fragmentation.
with mballoc, ext3 block reservation should be turned off and replaced
with the group-in-core-preallocation.
Has the new delayed allocation integrated with mballoc yet?
> I tried to add this call at the end of the ext4_da_get_block_write()
> function. This seems to fix the problem as the blocks are allocated
> contiguously on disk but the function seems to be called too many times
> so I think it is perhaps not the right place to call it.
>
> Who could look into this problem?
> I've got a few days off so I couldn't help more next days, but the
> problem is easily reproductible.
>
> Wouldn't this also explain why the compilebench results posted by Chris
> Mason are bad in some cases?
>
> Valérie
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html