Hi folks,
A new version erofs-utils 1.5 is available at:
git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git tags/v1.5
It mainly includes the following changes:
- (fsck.erofs) support filesystem extraction (Igor Ostapenko);
- support ztailpacking inline feature for compressed files (Yue Hu);
- (dump.erofs) support listing directories;
- more liberofs APIs (including iterate APIs) (me, Kelvin Zhang);
- use mtime to allow more control over the timestamps (David Anderson);
- switch to GPL-2.0+ OR Apache-2.0 dual license for liberofs;
- various bugfixes and cleanups;
A little bit delay this time for more than half a year. This release
mainly includes ztailpacking feature by Yue Hu which can inline the
tail pcluster with its inode metadata thus save space and a tail I/O,
it's highly recommended to be enabled if possible.
Apart from that, fsck.erofs now supports extracting filesystem, thanks
to Igor Ostapenko. There are other changes listed above.
In the end, I'd like to update the roadmap of EROFS since the last
update for the coming year:
https://lore.kernel.org/r/20211009061150.GA7479@hsiangkao-HP-ZHAN-66-Pro-G1
Thankfully many of them are finished during the past year.
1. Common stuffs:
- Switch to folios and enable large folios if possible in the next
cycles;
- Get rid of PG_error flag in Linux 5.20 (pending review);
- Explore byte-addressed rolling hash compression + deduplication since
on-disk format already supports such way but needs runtime tuning;
- LZ4 range dictionary support. We don't have enough manpower on this
yet, but hopefully it can have some progress in the coming year;
- Further code cleanups.
2. Container image use cases:
- Recently, we posted a article to introduce erofs over fscache
feature working with CNCF Dragonfly Nydus image service and give
some performance numbers.
https://d7y.io/blog/2022/06/06/evolution-of-nydus/
Our Alibaba kernel team are still working on several stuffs about
Nydus image service and fscache, including:
- Better flexible cache management, including repacking and blob
GC in order to make better use to the local cache database;
- Convert and run (e)stargz and others on the fly with fscache
feature. In the future, different formats are also able to be
merged in one fs tree:
https://github.com/dragonflyoss/image-service/pull/486
- Runtime decompression support over fscache;
- Blob cache sharing within the same trusted domain;
- Page cache sharing between different files with the same chunk;
- Enhanced convergent encryption to share chunk data in a trusted
domain and runtime verification;
- And other fscache/cachefiles common improvements like fallback
format, multiple daemons/dirs, FSDAX, etc.
- Apart from Nydus, it's planned to introduce a native fscache daemon
integrated in erofs-utils to mount EROFS, (e)stargz images from
network as well as provide fscache interfaces as liberofs APIs.
3. Embedded devices
- Yue Hu is currently working on a fragment-likewise feature, which
can merged tails or the whole files into a special inode in order to
minimize the space;
- Multi-threaded mkfs, fsck.erofs. I know someone is working on this
but I'm not sure the current progress. It's a bit delay but needs
to be resolved anyway.
Thanks,
Gao Xiang