Hello,
I am starting with ext4 development, so I have lot's of questions =)
1> About ext4 feature FLEX_BG, I see that it merges multiple
groups in 1 group, but I wonder how this makes sense...
In ext3 did each block group contain always 1 block?
If not, then how many?
And if not, how does that differ from FLEX_BG?
2> I see that the number of groups inside a flexible group is
given by 2^sb.s_log_groups_per_flex but:
a> Every single group is a flexible group or some are and some now?
b> This number is the same for all flexible groups?
3> Could anyone be kind enough to post the full path which is done
to find data in a given file?
I already found inode.i_block .... wonder what are the other links between
things, like for example between a group and inodes, etc...
thanks,
Felipe Monteiro de Carvalho
PS: I need to restrict each line to 80 chars because of gmane,
sorry for that =(
Hello,
I can now already answer some of my questions, thought others not:
> In ext3 did each block group contain always 1 block?
> If not, then how many?
This information is in ext4_super_block.s_blocks_per_group
> 2> I see that the number of groups inside a flexible group is
> given by 2^sb.s_log_groups_per_flex but:
> a> Every single group is a flexible group or some are and some now?
I am still very much interesting in knowing this! Are all groups flexible
groups when FLEX_BG feature is activated?
I found this image for how things relate to one another without flex_bg:
http://origin-ars.els-cdn.com/content/image/1-s2.0-S1742287612000357-gr1.jpg
Does anyone know if there is a simular image with flex_bg???
thanks,
Felipe Monteiro de Carvalho
On Sun, Jun 23, 2013 at 06:29:18AM +0000, Felipe Monteiro de Carvalho wrote:
>
> I am still very much interesting in knowing this! Are all groups flexible
> groups when FLEX_BG feature is activated?
>
> I found this image for how things relate to one another without flex_bg:
>
> http://origin-ars.els-cdn.com/content/image/1-s2.0-S1742287612000357-gr1.jpg
>
> Does anyone know if there is a simular image with flex_bg???
Historically, before flex_bg, the allocation bitmaps and inode tables
for a particular block group had to be located within that block
group, per the image that you've referenced above.
This had two difficulties. The first is that every 128MB (assuming a
4k blocksize and the default/maximum number of blocks per block
group), there would be file system metadata, thus breaking up the
maximum contiguous free space available for data blocks. The second
is that if e2fsck or resize2fs needed to relocate a metadata block, it
reduced the flexibility of where the new location of that metadata
block might be.
So the flex_bg feature relaxes this constraint. So if we can put the
metadata blocks anywhere, where do we put them? That's where
s_log_flex_bg_size in the superblock comes in. This is a default
policy used by mke2fs and for online resizing where we try to put the
allocation bitmaps and inode table blocks at the beginning of a flex
group, where flex group N is defined as blocks falling within the
range (N * flex_bg_size) through (((N + 1) * flex_bg_size) - 1).
Note though that this is not a mandatory layout; there may be times
where e2fsck or resize2fs may need to change where these blocks end up
living.
The other thing which we do with this flex_bg layout is that it
influences ext4's inode and block allocation hueristcs. For example,
ext4 will try to bias its allocation of directory blocks to try to
keep them within the first block group, and it will bias the
allocation of data blocks away from the first block group in a flex_bg
group.
Regards,
- Ted
On Sun, Jun 23, 2013 at 1:59 PM, Theodore Ts'o <[email protected]> wrote:
> Historically, before flex_bg, the allocation bitmaps and inode tables
> for a particular block group had to be located within that block
> group, per the image that you've referenced above.
Thanks, but I'm still not sure: Are all groups flexible groups when
FLEX_BG feature is activated?
If not, I wonder how a reader will recognize which groups were merged ...
thanks,
--
Felipe Monteiro de Carvalho
On Mon, Jul 01, 2013 at 05:23:13PM +0200, Felipe Monteiro de Carvalho wrote:
> On Sun, Jun 23, 2013 at 1:59 PM, Theodore Ts'o <[email protected]> wrote:
> > Historically, before flex_bg, the allocation bitmaps and inode tables
> > for a particular block group had to be located within that block
> > group, per the image that you've referenced above.
>
> Thanks, but I'm still not sure: Are all groups flexible groups when
> FLEX_BG feature is activated?
I'm not sure whant you mean by "flexible groups". There is no such
concept.
> If not, I wonder how a reader will recognize which groups were merged ...
And block groups don't get merged.
You should think of a flex block group as a collection of block
groups. (Just as a number of provinces make up country, or a
collection of towns make up a county, and a number of counties make up
a province.)
- Ted
On Mon, Jul 1, 2013 at 6:55 PM, Theodore Ts'o <[email protected]> wrote:
> I'm not sure whant you mean by "flexible groups". There is no such
> concept.
This wiki says that flex_bg stands for "Flexible Block Groups":
https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Flexible_Block_Groups
>> If not, I wonder how a reader will recognize which groups were merged ...
>
> And block groups don't get merged.
>
> You should think of a flex block group as a collection of block
> groups. (Just as a number of provinces make up country, or a
> collection of towns make up a county, and a number of counties make up
> a province.)
I'll try to be more specific.
When I am reading a ext4 file system and I find that there exists an
inode /etc/ with groupnr 32 and nodenr 260099, will this group 32 be
in the same position (distance in bytes from the beginning of the
disk) as in ext3? Or will the flex_bg make the first 16 groups fit
into the size that previously 1 group utilized? So group 32 in ext 4
would be in the physical position of group 2 in ext3.
If I have s_log_groups_per_flex=4 (seams the be the most popular
value), then each and every set of 16 groups will be groups into
flex_bgs or only some are grouped into flex_bgs?
Basically I have a reader for ext3 and I added EXTENT support and it
works now fine for the group 0, but for higher groups it is not
working and I'm trying to find out what I need to change ... I'm
trying to find what physically changed here... but it is a hard
battle. Much harder than EXTENT support was.
thanks,
--
Felipe Monteiro de Carvalho
On Wed, Jul 10, 2013 at 06:13:08PM +0200, Felipe Monteiro de Carvalho wrote:
>
> When I am reading a ext4 file system and I find that there exists an
> inode /etc/ with groupnr 32 and nodenr 260099, will this group 32 be
> in the same position (distance in bytes from the beginning of the
> disk) as in ext3? Or will the flex_bg make the first 16 groups fit
> into the size that previously 1 group utilized? So group 32 in ext 4
> would be in the physical position of group 2 in ext3.
>
> If I have s_log_groups_per_flex=4 (seams the be the most popular
> value), then each and every set of 16 groups will be groups into
> flex_bgs or only some are grouped into flex_bgs?
>
> Basically I have a reader for ext3 and I added EXTENT support and it
> works now fine for the group 0, but for higher groups it is not
> working and I'm trying to find out what I need to change ... I'm
> trying to find what physically changed here... but it is a hard
> battle. Much harder than EXTENT support was.
Your reader shouldn't care about the flex_bg feature. From a reader's
perspective, flex_bg's don't matter. Just ignore it. Just think in
terms of block groups, and you'll be find. The Flex_bg feature
matters for how our block allocation strategy works, and it influences
how online resizing works, and it tells e2fsck that it's ok for the
inode table for a particular block group to be located outside of its
own block group (which is where the "flexible" came from).
But from a read-only client is concerned, it doesn't matter. Just use
the block group descriptors to find the location of the inode table,
and just expunge the concept of flex_bg's from your mind entirely. Or
better yet, just use the libext2fs library, and don't try to reinvent
the wheel.....
- Ted
Hello,
That would be great, but then how to explain that
EXT4_FEATURE_INCOMPAT_FLEX_BG is present in
superblock^.s_feature_incompat
Which indicates that knowledge of this feature is necessary in the reader
--
Felipe Monteiro de Carvalho
On Thu, Jul 11, 2013 at 09:37:05AM +0200, Felipe Monteiro de Carvalho wrote:
> Hello,
>
> That would be great, but then how to explain that
> EXT4_FEATURE_INCOMPAT_FLEX_BG is present in
> superblock^.s_feature_incompat
>
> Which indicates that knowledge of this feature is necessary in the reader
That was because originally the Linux kernel implementation would
check to make sure the inode table and allocation bitmaps for block
group N were in fact located in block group N. If they were not, the
kernel would issue a lot of very scary warnings and mark the file
system as being corrupt when you tried to mount it.
But from a read-only implementation's perspective, the only thing you
need to know about the flex_bg feature is that inode table and
allocation bitmaps now have the __flexibility__ (hence the naming of
the file system feature "flex_bg") to be located outside of the block
group that they belong to.
The exact layout of how mke2fs and resize2fs will try to position the
inode tables is what is controlled by the flex_bg "size", where if the
flex_bg size is 16 block groups, we will try to locate the bg metadata
(i.e., inode tables plus allocation bitmaps) for blockgroups 0..15 in
bg 0, and the bg metadata for blockgroups 16..31 in bg 16, etc. This
is a "best efforts" sort of thing, and there cases where this may not
be tree (for example, off-line resizing, in particular an off-line
shrink may change this). So in the spirit of "be liberal in what you
accept, and conservative in what you receive", an implementation
should be prepared to deal with the inode table block and allocation
bitmaps being located anywhere in the file system. It is _likely_
that the metadata block for a flex_bg will be located in a flex_bg,
but it is not guaranteed.
As used in the last sentence above, the term "flex_bg" is also
shorthand to refer to the collection of block groups 0 through 15 as a
"flex_bg" and blockgroups "15..31" as a flex_bg. Yes, this is
confusing, although it's usually obvious from context whether
"flex_bg" is referring to the file system feature, or to a collection
of block groups.
The latter case is where where the allocation policy comes in, where
inodes which are located in the inode table corresponding to a flex_bg
consisting of block groups 0 through 15 will try to start allocating
directory blocks and extent tree blocks in block group 0, and data
blocks starting in block groups 1 and moving on through block group
15, and only then will we try to find another flex_bg to allocate the
data blocks.
The block allocation decisions and the layout of the inode table
blocks and allocation bitmaps only only matter if you are implementing
a read/write implementation of ext4, and they aren't even mandatory.
You could in theory create a read/write implementation that understood
the flex_bg feature, but used the layout and allocation algorithms
corresponding with ext3. This will result in a much less performant
implementation, and cause greater file system fragmentation, but it
would be valid in terms of e2fsck passing judgement on whether the
file system is consistent. Remember, the key word in "flex_bg" is
__flexibility__; it is what allows for more intelligent block
allocation algorithms and file system layouts.
Finally, can you please tell us what you are trying to do. From what
I can tell, you are implementing some kind of propetiary read-only
library to read ext4 file systems? Is this right? If so, can I
pursuade you not to make it be proprietary, so you can use the
libext2fs library? I've given you a lot of free advice and tutorials
in doing this, so it would be nice if you could reciprocate by telling
us what you are up to. Maybe we can help you with more targetted
advice if we knew what you were doing.
Thanks, regards,
- Ted
On Sun, Jun 23, 2013 at 1:59 PM, Theodore Ts'o <[email protected]> wrote:
> This had two difficulties. The first is that every 128MB (assuming a
> 4k blocksize and the default/maximum number of blocks per block
> group), there would be file system metadata, thus breaking up the
> maximum contiguous free space available for data blocks.
But the break up of contiguous free space still occurs, because each
group has a group descriptor in it's beginning, right?
--
Felipe Monteiro de Carvalho
On Mon, Jul 15, 2013 at 12:20:01PM +0200, Felipe Monteiro de Carvalho wrote:
> On Sun, Jun 23, 2013 at 1:59 PM, Theodore Ts'o <[email protected]> wrote:
> > This had two difficulties. The first is that every 128MB (assuming a
> > 4k blocksize and the default/maximum number of blocks per block
> > group), there would be file system metadata, thus breaking up the
> > maximum contiguous free space available for data blocks.
>
> But the break up of contiguous free space still occurs, because each
> group has a group descriptor in it's beginning, right?
The sparse_super feature restricts the number of group descriptors to
those block groups which are multiples of the nth root of 3, 5, or 7.
So a 4T file system, only has 22 superblocks and group descriptors out
of 32768 blok groups (at block groups: 1, 3, 5, 7, 9, 25, 27, 49, 81,
125, 243, 343, 625, 729, 2187, 2401, 3125, 6561, 15625, 16807, and
19683).
There is also the meta_bg feature, which defines yet another concept,
a meta_bg group, which is the set of block groups which can fit in a
single block group descriptor block. So for a 64-bit file system with
64-byte block group descriptors, you can fit 64 descriptors in a 4k
block. So the size of a meta_bg group is 64 block groups. (For a
file system w/o the 64-bit feature, the size of a meta_bg group is 128
block groups for file systems with 4k blocks.) With a file system
with a meta_bg feature enabled, starting with the nth s_first_meta_bg
(i.e., block group s_first_meta_bg * bg_per_meta_bg), the block groups
are located at the first, second, and last block groups in a meta_bg
group.
The reason for the meta_bg feature is the resize_inode uses indirect
blocks, which only works for 32-bit block numbers. So when we are
doing an online resize, once we expand the file system to the point
where we've run out of reserved bg descriptor blocks, or when we
expand beyond the 32-bit block number boundary, we enable the meta_bg
feature. The advantage of the meta_bg feature is that it's no longer
required to reserved bg descriptor blocks, or to move the inode table
when doing off-line resizes. The disadvantage of the meta_bg feature
is that the primary block groups are no longer contiguous, so it slows
down the time to mount a file system with a large number of bg
descriptor blocks which are laid out using the meta_bg feature.
Regards,
- Ted
P.S. And all of this is automatically handled by the ext2fs library. :-P
Hello,
Thanks. Now I think that I am getting closer to something more concrete.
My superblock says:
superblock^.s_blocks_per_group = 32768
superblock^.s_inodes_per_group = 8080
superblock^.s_log_block_size = 2 -> so each block has 4k bytes
And when reading the root "/" I obtain the following information for
the directory /bin/ : inode=8081
So I calculate that it should have:
groupnr=1 (because it is above 8080 inodes per group)
offset for groupnr 1 --> 4k * 32768 = 0x8000000
But I have utilized a hex editor (oh boy, it was hard to find one that
would read files over 4GB....) And the directory listing of /bin/ is
located in aprox. near 0x21B0000
As if each block group would have a size of 0x2000000 .... but this
size would be valid I would expect if
superblock^.s_blocks_per_group=8192 ...
Any idea what I missed here that the calculations went wrong?
thanks,
--
Felipe Monteiro de Carvalho
On Mon, Jul 15, 2013 at 06:09:03PM +0200, Felipe Monteiro de Carvalho wrote:
> Hello,
>
> Thanks. Now I think that I am getting closer to something more concrete.
>
> My superblock says:
>
> superblock^.s_blocks_per_group = 32768
> superblock^.s_inodes_per_group = 8080
> superblock^.s_log_block_size = 2 -> so each block has 4k bytes
>
> And when reading the root "/" I obtain the following information for
> the directory /bin/ : inode=8081
>
> So I calculate that it should have:
> groupnr=1 (because it is above 8080 inodes per group)
> offset for groupnr 1 --> 4k * 32768 = 0x8000000
>
> But I have utilized a hex editor (oh boy, it was hard to find one that
> would read files over 4GB....) And the directory listing of /bin/ is
> located in aprox. near 0x21B0000
>
> As if each block group would have a size of 0x2000000 .... but this
> size would be valid I would expect if
> superblock^.s_blocks_per_group=8192 ...
>
> Any idea what I missed here that the calculations went wrong?
You're forgetting to read the inode structure. So take the inode
number, determine that it's in the inode table fragment which belongs
to block group #1, then look up in the block group descriptors the
starting block of the inode table for bg #1, then look up the inode
size in the superblock, to determine the block number and offset to
read the inode structure.
Then you need to look at the inode structure to map logical block #0,
to the physical block number where the directory entries are located.
I suggest you RTFS in libext2fs....
- Ted
Hello,
Sorry for my dumb questions, but please, just 1 more =D
In another separate case, and a different partition being read too, I have:
/usr/ directory with information:
inode = 261837
superblock^.s_blocks_per_group = 32768
superblock^.s_inodes_per_group = 8128
superblock^.s_log_block_size = 2 -> so each block has 4k bytes
superblock^.s_inodes_per_group = 8128
number of block groups = (superblock^.s_blocks_count +
superblock^.s_blocks_per_group-1) div superblock^.s_blocks_per_group =
63
So I calculate (inodenr-1)/(inodes per group)=32 -> so the group nr
for inode /usr/ is 32
So next I decide to attempt to read the group descriptor for group 32
But my table of group descriptors read from group nr 0 is all correct
between group nr 0 and 31, but group nr 32 to 63 are filled with
zeroes.
>From what I had understood, the group descriptor in group zero should
contain all group descriptors from all groups, and subsequent
occurences of the group descriptor table would be backups.
Clearly it is not like that ... so I suppose that in group nr 32 I'll
find a table of group descriptors for groups between 32 and 63 or
something like that ... or maybe they did not fill in that block and
are somewhere else in group 0 .... any ideas?
full table of group descriptors read from group 0:
memoDebugOut
Group Descriptor Nr=0 bg_block_bitmap_lo=501 bg_inode_bitmap_lo=517
bg_inode_table_lo=533 bg_free_blocks_count_lo=21463
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=1232 bg_flags=4
Group Descriptor Nr=1 bg_block_bitmap_lo=503 bg_inode_bitmap_lo=519
bg_inode_table_lo=1549 bg_free_blocks_count_lo=47
bg_free_inodes_count_lo=23 bg_used_dirs_count_lo=633 bg_flags=4
Group Descriptor Nr=2 bg_block_bitmap_lo=505 bg_inode_bitmap_lo=521
bg_inode_table_lo=2565 bg_free_blocks_count_lo=2
bg_free_inodes_count_lo=8128 bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=3 bg_block_bitmap_lo=507 bg_inode_bitmap_lo=523
bg_inode_table_lo=3581 bg_free_blocks_count_lo=3410
bg_free_inodes_count_lo=8128 bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=4 bg_block_bitmap_lo=509 bg_inode_bitmap_lo=525
bg_inode_table_lo=4597 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=8128 bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=5 bg_block_bitmap_lo=511 bg_inode_bitmap_lo=527
bg_inode_table_lo=5613 bg_free_blocks_count_lo=16
bg_free_inodes_count_lo=8128 bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=6 bg_block_bitmap_lo=513 bg_inode_bitmap_lo=529
bg_inode_table_lo=6629 bg_free_blocks_count_lo=26803
bg_free_inodes_count_lo=8128 bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=7 bg_block_bitmap_lo=515 bg_inode_bitmap_lo=531
bg_inode_table_lo=7645 bg_free_blocks_count_lo=27542
bg_free_inodes_count_lo=8128 bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=8 bg_block_bitmap_lo=524288
bg_inode_bitmap_lo=524304 bg_inode_table_lo=524320
bg_free_blocks_count_lo=20867 bg_free_inodes_count_lo=0
bg_used_dirs_count_lo=573 bg_flags=4
Group Descriptor Nr=9 bg_block_bitmap_lo=524290
bg_inode_bitmap_lo=524306 bg_inode_table_lo=525336
bg_free_blocks_count_lo=171 bg_free_inodes_count_lo=0
bg_used_dirs_count_lo=423 bg_flags=4
Group Descriptor Nr=10 bg_block_bitmap_lo=524292
bg_inode_bitmap_lo=524308 bg_inode_table_lo=526352
bg_free_blocks_count_lo=0 bg_free_inodes_count_lo=499
bg_used_dirs_count_lo=1666 bg_flags=4
Group Descriptor Nr=11 bg_block_bitmap_lo=524294
bg_inode_bitmap_lo=524310 bg_inode_table_lo=527368
bg_free_blocks_count_lo=32243 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=12 bg_block_bitmap_lo=524296
bg_inode_bitmap_lo=524312 bg_inode_table_lo=528384
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=13 bg_block_bitmap_lo=524298
bg_inode_bitmap_lo=524314 bg_inode_table_lo=529400
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=14 bg_block_bitmap_lo=524300
bg_inode_bitmap_lo=524316 bg_inode_table_lo=530416
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=15 bg_block_bitmap_lo=524302
bg_inode_bitmap_lo=524318 bg_inode_table_lo=531432
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=16 bg_block_bitmap_lo=1048576
bg_inode_bitmap_lo=1048592 bg_inode_table_lo=1048608
bg_free_blocks_count_lo=20418 bg_free_inodes_count_lo=0
bg_used_dirs_count_lo=654 bg_flags=4
Group Descriptor Nr=17 bg_block_bitmap_lo=1048578
bg_inode_bitmap_lo=1048594 bg_inode_table_lo=1049624
bg_free_blocks_count_lo=12 bg_free_inodes_count_lo=0
bg_used_dirs_count_lo=2256 bg_flags=4
Group Descriptor Nr=18 bg_block_bitmap_lo=1048580
bg_inode_bitmap_lo=1048596 bg_inode_table_lo=1050640
bg_free_blocks_count_lo=0 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=19 bg_block_bitmap_lo=1048582
bg_inode_bitmap_lo=1048598 bg_inode_table_lo=1051656
bg_free_blocks_count_lo=0 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=20 bg_block_bitmap_lo=1048584
bg_inode_bitmap_lo=1048600 bg_inode_table_lo=1052672
bg_free_blocks_count_lo=17447 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=21 bg_block_bitmap_lo=1048586
bg_inode_bitmap_lo=1048602 bg_inode_table_lo=1053688
bg_free_blocks_count_lo=7625 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=22 bg_block_bitmap_lo=1048588
bg_inode_bitmap_lo=1048604 bg_inode_table_lo=1054704
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=23 bg_block_bitmap_lo=1048590
bg_inode_bitmap_lo=1048606 bg_inode_table_lo=1055720
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=24 bg_block_bitmap_lo=1572864
bg_inode_bitmap_lo=1572880 bg_inode_table_lo=1572896
bg_free_blocks_count_lo=20845 bg_free_inodes_count_lo=1
bg_used_dirs_count_lo=790 bg_flags=4
Group Descriptor Nr=25 bg_block_bitmap_lo=1572866
bg_inode_bitmap_lo=1572882 bg_inode_table_lo=1573912
bg_free_blocks_count_lo=0 bg_free_inodes_count_lo=0
bg_used_dirs_count_lo=188 bg_flags=4
Group Descriptor Nr=26 bg_block_bitmap_lo=1572868
bg_inode_bitmap_lo=1572884 bg_inode_table_lo=1574928
bg_free_blocks_count_lo=118 bg_free_inodes_count_lo=0
bg_used_dirs_count_lo=1314 bg_flags=4
Group Descriptor Nr=27 bg_block_bitmap_lo=1572870
bg_inode_bitmap_lo=1572886 bg_inode_table_lo=1575944
bg_free_blocks_count_lo=30412 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=28 bg_block_bitmap_lo=1572872
bg_inode_bitmap_lo=1572888 bg_inode_table_lo=1576960
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=29 bg_block_bitmap_lo=1572874
bg_inode_bitmap_lo=1572890 bg_inode_table_lo=1577976
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=30 bg_block_bitmap_lo=1572876
bg_inode_bitmap_lo=1572892 bg_inode_table_lo=1578992
bg_free_blocks_count_lo=32768 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=7
Group Descriptor Nr=31 bg_block_bitmap_lo=1572878
bg_inode_bitmap_lo=1572894 bg_inode_table_lo=1580008
bg_free_blocks_count_lo=14655 bg_free_inodes_count_lo=8128
bg_used_dirs_count_lo=0 bg_flags=5
Group Descriptor Nr=32 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=33 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=34 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=35 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=36 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=37 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=38 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=39 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=40 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=41 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=42 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=43 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=44 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=45 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=46 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=47 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=48 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=49 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=50 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=51 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=52 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=53 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=54 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=55 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=56 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=57 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=58 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=59 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=60 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=61 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
Group Descriptor Nr=62 bg_block_bitmap_lo=0 bg_inode_bitmap_lo=0
bg_inode_table_lo=0 bg_free_blocks_count_lo=0
bg_free_inodes_count_lo=0 bg_used_dirs_count_lo=0 bg_flags=0
thank you very much,
--
Felipe Monteiro de Carvalho
Can you please run dumpe2fs on the file system in question?
That should answer your questions. If not, then send me the output.
The output of dumpe2fs is also much easier for me to parse than your
ad-hoc data dumps....
- Ted
wow, that's a pretty amazing tool, didn't know about it.
Thought I still don't see an answer to this particular last question there.
thanks,
--
Felipe Monteiro de Carvalho
On Mon, Jul 15, 2013 at 08:18:16PM +0200, Felipe Monteiro de Carvalho wrote:
> wow, that's a pretty amazing tool, didn't know about it.
Are you familiar with debugfs? I'd also strongly suggest that you try
using that program to try to understand the file system layout.
>But my table of group descriptors read from group nr 0 is all correct
>between group nr 0 and 31, but group nr 32 to 63 are filled with
>zeroes.
> Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
The file system does not have the 64-bit feature set. Hence, the
layout of the block grup descriptor is struct ext2_group_desc, and not
struct ext4_group_desc.
I'm guessing that you were using the ext4_group_desc structure (which
is 64 bytes) instead of the ext2_group_desc structure (which is 32
bytes).
I would suggest that you match up the contents of your in-program data
structures, and compare it to the outputs of debugfs and dumpe2fs.
- Ted
wow, that was a genial catch, thank you very much =)
With this tip I managed to find what was wrong in the code and then
fixed some other stuff and now it seams that my ext4 reader is fully
working correctly =D
It did come to my mind the size of group descriptors many days ago,
but I had supposed that the _hi elements would be filled with zeroes,
not that they would be completely removed.
Also good to know about debugfs and dumpe2fs
thanks again for all the help and sorry for any disturbance, =)
--
Felipe Monteiro de Carvalho
