Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 13 Jun 2022 04:31:34 +0800
From:   Gao Xiang <xiang@kernel.org>
To:     linux-erofs@lists.ozlabs.org
Cc:     linux-fsdevel@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>,
        kernel-team@android.com,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jonathan Corbet <corbet@lwn.net>
Subject: [ANNOUNCE] erofs-utils: release 1.5
Message-ID: <YqZNJpgQ+xLSHBqK@debian>
Mail-Followup-To: linux-erofs@lists.ozlabs.org,
        linux-fsdevel@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>,
        kernel-team@android.com,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jonathan Corbet <corbet@lwn.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Precedence: bulk

Hi folks,

A new version erofs-utils 1.5 is available at:
git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git tags/v1.5

It mainly includes the following changes:
   - (fsck.erofs) support filesystem extraction (Igor Ostapenko);
   - support ztailpacking inline feature for compressed files (Yue Hu);
   - (dump.erofs) support listing directories;
   - more liberofs APIs (including iterate APIs) (me, Kelvin Zhang);
   - use mtime to allow more control over the timestamps (David Anderson);
   - switch to GPL-2.0+ OR Apache-2.0 dual license for liberofs;
   - various bugfixes and cleanups;


A little bit delay this time for more than half a year.  This release
mainly includes ztailpacking feature by Yue Hu which can inline the
tail pcluster with its inode metadata thus save space and a tail I/O,
it's highly recommended to be enabled if possible.

Apart from that, fsck.erofs now supports extracting filesystem, thanks
to Igor Ostapenko.  There are other changes listed above.


In the end, I'd like to update the roadmap of EROFS since the last
update for the coming year:

https://lore.kernel.org/r/20211009061150.GA7479@hsiangkao-HP-ZHAN-66-Pro-G1

Thankfully many of them are finished during the past year.


1. Common stuffs:

 - Switch to folios and enable large folios if possible in the next
   cycles;

 - Get rid of PG_error flag in Linux 5.20 (pending review);

 - Explore byte-addressed rolling hash compression + deduplication since
   on-disk format already supports such way but needs runtime tuning;

 - LZ4 range dictionary support.  We don't have enough manpower on this
   yet, but hopefully it can have some progress in the coming year;

 - Further code cleanups.


2. Container image use cases:

 - Recently, we posted a article to introduce erofs over fscache
   feature working with CNCF Dragonfly Nydus image service and give
   some performance numbers.

     https://d7y.io/blog/2022/06/06/evolution-of-nydus/

   Our Alibaba kernel team are still working on several stuffs about
   Nydus image service and fscache, including:

    - Better flexible cache management, including repacking and blob
      GC in order to make better use to the local cache database;

    - Convert and run (e)stargz and others on the fly with fscache
      feature.  In the future, different formats are also able to be
      merged in one fs tree:
      https://github.com/dragonflyoss/image-service/pull/486

    - Runtime decompression support over fscache;

    - Blob cache sharing within the same trusted domain;

    - Page cache sharing between different files with the same chunk;

    - Enhanced convergent encryption to share chunk data in a trusted
      domain and runtime verification;

    - And other fscache/cachefiles common improvements like fallback
      format, multiple daemons/dirs, FSDAX, etc.

 - Apart from Nydus, it's planned to introduce a native fscache daemon
   integrated in erofs-utils to mount EROFS, (e)stargz images from
   network as well as provide fscache interfaces as liberofs APIs.


3. Embedded devices

 - Yue Hu is currently working on a fragment-likewise feature, which
   can merged tails or the whole files into a special inode in order to
   minimize the space;

 - Multi-threaded mkfs, fsck.erofs.  I know someone is working on this
   but I'm not sure the current progress.  It's a bit delay but needs
   to be resolved anyway.


Thanks,
Gao Xiang