Hi Linus,
Could you consider this pull request for 5.13-rc1?
In this cycle, we would like to introduce a new feature called big
pcluster so EROFS can compress file data into more than 1 fs block
and different pcluster size can be selected for each (sub-)files by
design. The current EROFS test results on my laptop are [1]:
Testscript: erofs-openbenchmark
Testdata: enwik9 (1000000000 bytes)
________________________________________________________________
| file system | size | seq read | rand read | rand9m read |
|_______________|___________|_ MiB/s __|__ MiB/s __|___ MiB/s ___|
|___erofs_4k____|_556879872_|_ 781.4 __|__ 55.3 ___|___ 25.3 ___|
|___erofs_16k___|_452509696_|_ 864.8 __|_ 123.2 ___|___ 20.8 ___|
|___erofs_32k___|_415223808_|_ 899.8 __|_ 105.8 _*_|___ 16.8 ____|
|___erofs_64k___|_393814016_|_ 906.6 __|__ 66.6 _*_|___ 11.8 ____|
|__squashfs_8k__|_556191744_|_ 64.9 __|__ 19.3 ___|____ 9.1 ____|
|__squashfs_16k_|_502661120_|_ 98.9 __|__ 38.0 ___|____ 9.8 ____|
|__squashfs_32k_|_458784768_|_ 115.4 __|__ 71.6 _*_|___ 10.0 ____|
|_squashfs_128k_|_398204928_|_ 257.2 __|_ 253.8 _*_|___ 10.9 ____|
|____ext4_4k____|____()_____|_ 786.6 __|__ 28.6 ___|___ 27.8 ____|
, which has been verified but I'd like warn it as experimental for
a while. This matchs erofs-utils dev branch and I'll also release
a new userspace version for this later.
Apart from that, several improvements are also included: e.g. complete
a missing case for inplace I/O, optimize endio decompression logic for
non-atomic contexts and support adjustable sliding window size, ...
In addition to those, there are some cleanups as always.
All commits have been tested and have been in linux-next for about 2
weeks and I've just confirmed with 5.12 release. This merges cleanly
with master.
[1] https://lore.kernel.org/r/[email protected]
Thanks,
Gao Xiang
The following changes since commit a5e13c6df0e41702d2b2c77c8ad41677ebb065b3:
Linux 5.12-rc5 (2021-03-28 15:48:16 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git tags/erofs-for-5.13-rc1
for you to fetch changes up to 8e6c8fa9f2e95c88a642521a5da19a8e31748846:
erofs: enable big pcluster feature (2021-04-10 03:20:19 +0800)
----------------------------------------------------------------
Changes since last update:
- avoid memory failure when applying rolling decompression;
- optimize endio decompression logic for non-atomic contexts;
- complete a missing case which can be safely selected for inplace
I/O and thus decreasing more memory footprint;
- check unsupported on-disk inode i_format strictly;
- support adjustable lz4 sliding window size to decrease runtime
memory footprint;
- support on-disk compression configurations;
- support big pcluster decompression;
- several code cleanups / spelling correction.
----------------------------------------------------------------
Gao Xiang (15):
erofs: complete a missing case for inplace I/O
erofs: add unsupported inode i_format check
erofs: introduce erofs_sb_has_xxx() helpers
erofs: introduce on-disk lz4 fs configurations
erofs: add on-disk compression configurations
erofs: reserve physical_clusterbits[]
erofs: introduce multipage per-CPU buffers
erofs: introduce physical cluster slab pools
erofs: fix up inplace I/O pointer for big pcluster
erofs: add big physical cluster definition
erofs: adjust per-CPU buffers according to max_pclusterblks
erofs: support parsing big pcluster compress indexes
erofs: support parsing big pcluster compact indexes
erofs: support decompress big pcluster for lz4 backend
erofs: enable big pcluster feature
Huang Jianan (4):
erofs: avoid memory allocation failure during rolling decompression
erofs: use workqueue decompression for atomic contexts only
erofs: use sync decompression for atomic contexts only
erofs: support adjust lz4 history window size
Ruiqi Gong (1):
erofs: Clean up spelling mistakes found in fs/erofs
Yue Hu (1):
erofs: don't use erofs_map_blocks() any more
fs/erofs/Kconfig | 14 ---
fs/erofs/Makefile | 2 +-
fs/erofs/data.c | 19 +---
fs/erofs/decompressor.c | 272 ++++++++++++++++++++++++++++++------------------
fs/erofs/erofs_fs.h | 54 ++++++++--
fs/erofs/inode.c | 7 ++
fs/erofs/internal.h | 86 ++++++++++-----
fs/erofs/pcpubuf.c | 148 ++++++++++++++++++++++++++
fs/erofs/super.c | 148 +++++++++++++++++++++++++-
fs/erofs/utils.c | 12 ---
fs/erofs/zdata.c | 254 +++++++++++++++++++++++++++++---------------
fs/erofs/zdata.h | 14 +--
fs/erofs/zmap.c | 164 +++++++++++++++++++++++------
13 files changed, 887 insertions(+), 307 deletions(-)
create mode 100644 fs/erofs/pcpubuf.c
The pull request you sent on Mon, 26 Apr 2021 18:57:34 +0800:
> git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git tags/erofs-for-5.13-rc1
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/b5b3097d9cbb1eb3df0ade9507585e6e9e3b2385
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html