From: Tao Ma <tm@tao.ma>
Subject: [PATCH V6 00/23] ext4: Add inline data support.
Date: Fri, 14 Sep 2012 19:18:32 +0800
Message-ID: <1347621512-3660-1-git-send-email-tm@tao.ma>
To: linux-ext4@vger.kernel.org
Sender: linux-ext4-owner@vger.kernel.org

Hi list,

this is another try for the inline data support for ext4. The main
change in this version is that now we only store the 4 bytes for
parent's inode number and both "." and ".." are removed which
is suggested by Ted.

I have runned all the xfstests cases and fixes the bugs it found.
Hope it is stable now for the inclusion.

So in genernal, I have runned several test cases and try to prove
that it works.

A fsstats shows the directory information for my home.
(The test tools can be found here. http://www.pdsi-scidac.org/fsstats/.
Thanks goes to Andreas Dilger).

directory size (entries): Range of entries, count of directories in range, number of dirs in range as a % of total num of dirs, number of dirs in this range or smaller as a % total number of dirs, total entries in range, number of entries in range as a % of total number of entries, number of entries in this range or smaller as a % of total number of entries.
  count=13671 avg=11.58 ents
  min=0.00 ents max=4562.00 ents
  [   0-   1 ents]:  5198 (38.02%) ( 38.02% cumulative)   3322.00 ents ( 2.10%) (  2.10% cumulative) 
  [   2-   3 ents]:  2246 (16.43%) ( 54.45% cumulative)   5477.00 ents ( 3.46%) (  5.56% cumulative) 
  [   4-   7 ents]:  2266 (16.58%) ( 71.03% cumulative)  12037.00 ents ( 7.60%) ( 13.16% cumulative) 
  [   8-  15 ents]:  1877 (13.73%) ( 84.76% cumulative)  20269.00 ents (12.80%) ( 25.96% cumulative) 
  [  16-  31 ents]:  1126 ( 8.24%) ( 92.99% cumulative)  24196.00 ents (15.28%) ( 41.24% cumulative) 
  [  32-  63 ents]:   554 ( 4.05%) ( 97.04% cumulative)  23772.00 ents (15.01%) ( 56.26% cumulative) 
  [  64- 127 ents]:   234 ( 1.71%) ( 98.76% cumulative)  20919.00 ents (13.21%) ( 69.47% cumulative) 
  [ 128- 255 ents]:   119 ( 0.87%) ( 99.63% cumulative)  20654.00 ents (13.05%) ( 82.52% cumulative) 
  [ 256- 511 ents]:    40 ( 0.29%) ( 99.92% cumulative)  12941.00 ents ( 8.17%) ( 90.69% cumulative) 
  [ 512-1023 ents]:     6 ( 0.04%) ( 99.96% cumulative)   4147.00 ents ( 2.62%) ( 93.31% cumulative) 
  [1024-2047 ents]:     3 ( 0.02%) ( 99.99% cumulative)   3487.00 ents ( 2.20%) ( 95.51% cumulative) 
  [2048-4095 ents]:     1 ( 0.01%) ( 99.99% cumulative)   2544.00 ents ( 1.61%) ( 97.12% cumulative) 
  [4096-8191 ents]:     1 ( 0.01%) (100.00% cumulative)   4562.00 ents ( 2.88%) (100.00% cumulative) 

So with inode size equals 256, more than 50% directory can be inlined.

I have also runned several other test cases to prove the peroformance gain,
And I'd like to describe the result here. For more test details, please see below.

For a dir iteration test, if most of the dirs are inlined, we can see a 10 fold
speed up(240ms vs. 2250ms) in dir iteration(try to find a not-exist file). Yes,
that is expected. ;)

With a normal kernel tar and directory iteration test, we can find a 1.5% space
saving(with cluster size=4096, I would imagine a better result with bigalloc enabled).
As for the directory iteration, we can see a 22.4% speedup.

I ran postmark for 2 senarios, one with file size between 11 and 500 which is good
for inline data test, and another with file size between 100 and 50000. We can see
a 32.8%(49s vs. 73s) improvement for the first one, and 3.6%(805s vs. 835s) for the
latter.

The final test case is the fs_mark. I ran the different sync modes, and we can see
more than 100% increase if we does the sync and this is really a huge win.

1. normal test case of taring a kernel source and find a file:
time tar zxvf linux-3.4.tar.gz -C $MNT_DIR
df -hm|grep $MNT_DIR
umount $MNT_DIR
mount -t ext4 $DEVICE $MNT_DIR
time find $MNT_DIR -name a

inline_data result
in find real 570ms
in find sys 80ms
in find user 30ms

normal result
in find real 760ms
in find sys 110ms
in find user 30ms

inline_data + journal result
in find real 270ms
in find sys 80ms
in find user 30ms

normal journal result
in find real 590ms
in find sys 120ms
in find user 40ms

2. postmark test case 1:
set location $MNT_DIR
set subdirectories 2570
set size 11 500
set transactions 51110
set number 977770

with inline data result:
Time:
	56 seconds total
	2 seconds of transactions (25555 per second)

Files:
	1003234 created (17914 per second)
		Creation alone: 977770 files (29629 per second)
		Mixed with transactions: 25464 files (12732 per second)
	25561 read (12780 per second)
	25548 appended (12774 per second)
	1003234 deleted (17914 per second)
		Deletion alone: 977588 files (46551 per second)
		Mixed with transactions: 25646 files (12823 per second)

Data:
	6.25 megabytes read (114.27 kilobytes per second)
	247.18 megabytes written (4.41 megabytes per second)

Without inline data:
Time:
	71 seconds total
	24 seconds of transactions (2129 per second)

Files:
	1003234 created (14130 per second)
		Creation alone: 977770 files (31540 per second)
		Mixed with transactions: 25464 files (1061 per second)
	25561 read (1065 per second)
	25548 appended (1064 per second)
	1003234 deleted (14130 per second)
		Deletion alone: 977588 files (61099 per second)
		Mixed with transactions: 25646 files (1068 per second)

Data:
	6.25 megabytes read (90.13 kilobytes per second)
	247.18 megabytes written (3.48 megabytes per second)

postmark test 2:
set location $MNT_DIR
set subdirectories 2570
set size 100 50000
set transactions 51110
set number 977770

with inline data:
Time:
	805 seconds total
	263 seconds of transactions (194 per second)

Files:
	1003213 created (1246 per second)
		Creation alone: 977770 files (1975 per second)
		Mixed with transactions: 25443 files (96 per second)
	25597 read (97 per second)
	25513 appended (97 per second)
	1003213 deleted (1246 per second)
		Deletion alone: 977546 files (20798 per second)
		Mixed with transactions: 25667 files (97 per second)

without inline data:
Time:
	835 seconds total
	289 seconds of transactions (176 per second)

Files:
	1003213 created (1201 per second)
		Creation alone: 977770 files (2011 per second)
		Mixed with transactions: 25443 files (88 per second)
	25597 read (88 per second)
	25513 appended (88 per second)
	1003213 deleted (1201 per second)
		Deletion alone: 977546 files (16292 per second)
		Mixed with transactions: 25667 files (88 per second)

3. fs_mark test:
fs_mark -d $MNT_DIR -D 256 -s 256 -n 10000 -t 256 -S [0|1|2]

                         inline+nojournal noinline+nojournal
Files/sec for sync 0     69931.1          69305.7
Files/sec for sync 1     40674.0          13975.2
Files/sec for sync 2     45786.8          16154.8

                         inline+journal   noinline+journal
Files/sec for sync 0     63523.4          61092.1
Files/sec for sync 1     45325.4          20685.5
Files/sec for sync 2     41140.1          15908.6

git diff --stat
 fs/ext4/Makefile  |    2 +-
 fs/ext4/dir.c     |   39 +-
 fs/ext4/ext4.h    |   90 +++-
 fs/ext4/extents.c |   22 +-
 fs/ext4/ialloc.c  |    4 +
 fs/ext4/inline.c  | 1827 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/ext4/inode.c   |  237 ++++++--
 fs/ext4/namei.c   |  463 +++++++++-----
 fs/ext4/xattr.c   |   87 ++-
 fs/ext4/xattr.h   |  277 ++++++++
 10 files changed, 2783 insertions(+), 265 deletions(-)

any comments are welcomed.

Thanks
Tao