Hi Linus,
Could you consider this pull request for 5.16-rc1?
There are some new features available for this cycle. Firstly, EROFS
LZMA algorithm support, specifically called MicroLZMA, is available
as an option for embedded devices, LiveCDs and/or as the secondary
auxiliary compression algorithm besides the primary algorithm in one
file.
In order to better support the LZMA fixed-sized output compression,
especially for 4KiB pcluster size (which has lowest memory pressure
thus useful for memory-sensitive scenarios), Lasse introduced a new
LZMA header/container format called MicroLZMA to minimize the original
LZMA1 header (for example, we don't need to waste 4-byte dictionary
size and another 8-byte uncompressed size, which can be calculated by
fs directly, for each pcluster.) and enable EROFS fixed-sized output
compression. Note that MicroLZMA can also be later used by other
things in addition to EROFS too where wasting minimal amount of space
for headers is important and it can be only compiled by enabling
XZ_DEC_MICROLZMA. MicroLZMA has been supported by the latest upstream
XZ embedded [1] & XZ utils [2], apply the latest related XZ embedded
upstream patches by the XZ author Lasse here.
Secondly, multiple device is also supported in this cycle, which is
designed for multi-layer container images. By working together with
inter-layer data deduplication and compression, we can achieve the
next high-performance container image solution. Our team will announce
the new Nydus container image service [3] implementation with new RAFS
v6 (EROFS-compatible) format in Open Source Summit 2021 China [4] soon.
Besides, the secondary compression head support and readmore
decompression strategy are also included in this cycle. There are also
some minor bugfixes and cleanups, as always.
All commits have been tested and have been in linux-next. This merges
cleanly with master.
[1] https://git.tukaani.org/?p=xz-embedded.git
[2] https://git.tukaani.org/?p=xz.git
[3] https://github.com/dragonflyoss/image-service
[4] https://kccncosschn21.sched.com/event/pcdL/erofsdaelsju-nanojdyags-erofs-what-are-we-doing-now-for-containers-xiang-gao-alibaba
Thanks,
Gao Xiang
The following changes since commit 9e1ff307c779ce1f0f810c7ecce3d95bbae40896:
Linux 5.15-rc4 (2021-10-03 14:08:47 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git tags/erofs-for-5.16-rc1
for you to fetch changes up to a0961f351d82d43ab0b845304caa235dfe249ae9:
erofs: don't trigger WARN() when decompression fails (2021-10-31 21:00:28 +0800)
----------------------------------------------------------------
Changes since last update:
- support multiple devices for multi-layer container images;
- support the secondary compression head;
- support readmore decompression strategy;
- support new LZMA algorithm (specifically called MicroLZMA);
- some bugfixes & cleanups.
----------------------------------------------------------------
Gao Xiang (9):
erofs: decouple basic mount options from fs_context
erofs: add multiple device support
erofs: get compression algorithms directly on mapping
erofs: introduce the secondary compression head
erofs: introduce readmore decompression strategy
erofs: rename some generic methods in decompressor
erofs: lzma compression support
erofs: get rid of ->lru usage
erofs: don't trigger WARN() when decompression fails
Lasse Collin (5):
lib/xz: Avoid overlapping memcpy() with invalid input with in-place decompression
lib/xz: Validate the value before assigning it to an enum variable
lib/xz: Move s->lzma.len = 0 initialization to lzma_reset()
lib/xz: Add MicroLZMA decoder
lib/xz, lib/decompress_unxz.c: Fix spelling in comments
Yue Hu (1):
erofs: remove the fast path of per-CPU buffer decompression
Documentation/filesystems/erofs.rst | 12 +-
fs/erofs/Kconfig | 40 +++--
fs/erofs/Makefile | 1 +
fs/erofs/compress.h | 28 ++--
fs/erofs/data.c | 73 +++++++--
fs/erofs/decompressor.c | 139 ++++++-----------
fs/erofs/decompressor_lzma.c | 290 ++++++++++++++++++++++++++++++++++++
fs/erofs/erofs_fs.h | 73 ++++++---
fs/erofs/inode.c | 2 +-
fs/erofs/internal.h | 105 +++++++++++--
fs/erofs/pcpubuf.c | 6 +-
fs/erofs/super.c | 231 ++++++++++++++++++++++------
fs/erofs/utils.c | 19 ++-
fs/erofs/xattr.c | 4 +-
fs/erofs/zdata.c | 175 +++++++++++++++-------
fs/erofs/zdata.h | 7 -
fs/erofs/zmap.c | 65 +++++---
include/linux/xz.h | 106 +++++++++++++
include/trace/events/erofs.h | 2 +-
lib/decompress_unxz.c | 10 +-
lib/xz/Kconfig | 13 ++
lib/xz/xz_dec_lzma2.c | 182 +++++++++++++++++++++-
lib/xz/xz_dec_stream.c | 6 +-
lib/xz/xz_dec_syms.c | 9 +-
lib/xz/xz_private.h | 3 +
25 files changed, 1281 insertions(+), 320 deletions(-)
create mode 100644 fs/erofs/decompressor_lzma.c
The pull request you sent on Mon, 1 Nov 2021 08:40:32 +0800:
> git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git tags/erofs-for-5.16-rc1
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/67a135b80eb75b62d92a809b0246e70524f69b89
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html