Received: by 2002:a05:6358:d09b:b0:dc:cd0c:909e with SMTP id jc27csp2104695rwb; Wed, 30 Nov 2022 02:20:50 -0800 (PST) X-Google-Smtp-Source: AA0mqf6F/rtLBMENS7ipb5EKnUP7ZLY/gGqbqx0jhbXiI6FjZ952HUIM6QIhzPMvFmAmJK44qg/c X-Received: by 2002:a17:906:4351:b0:78d:513d:f447 with SMTP id z17-20020a170906435100b0078d513df447mr39097910ejm.708.1669803650135; Wed, 30 Nov 2022 02:20:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669803650; cv=none; d=google.com; s=arc-20160816; b=GQXqf6Q9yBRWLecuQSwNhSOQIovbX+fVttTvjKoUFq31nuUL/LvU/dRX9KT/exiJU8 h1MIFnCfYVN5rV1johvXRsSg+9xrpeufExDNt3pBrVvBES696TLyO9NSta6QXEH5crhP 3VHanXN2TowPCB1hljLLwNQPlVyDlWeWlgpj+Mul5/whLRAJARByIX5eHwe+ynHRQpp0 OuQRVpK4wzizotAiixJ1E2tAGSQhxuVNXXnmOdkjp2iIpqoftNhFh0RQZm1h0EPnCBiB KLeBjjzwkzzg4ZI7kJbBl/CdSZ6/tGqtGZScXQ2AN3DZAVTsJIysYbqps12Sas6fs9A7 TqTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=zg7cZb5nT7GSKDl8+pqxmWT78tqpAHGaKOGrEy9tk+M=; b=Lap4xo76THshi4dTJIEZmlicr9xS46LdLt0q6ksXmcCC7LbymvTzP8vaf9zRNiJuRj xbDzj4ip5ZycxudH2QyNmlJzpmBPICgYi0T8b+JRhMjVkM5m8ofJdD7o504XSQW7YuK1 Y+cslhNVzGkkDvduHlS0v/6/YQXQZRJlfhmJ/WaneAWLw+rRigNwf6IPwGhA7ijuljLu GG79Ft7kQGtrKtvB9NwnIQrciNLxKNVf+z//C8r5BEY6R7YjcBuAynZqDhuX++BCarka afkbtWiQQJK4XPi0iP5ylKFL9stNgvVh5g1ygR7HKuTF/cCdrEbHflnMAHYnY4GOHb+R Uufg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i6-20020a1709064fc600b0078d93325645si1083125ejw.405.2022.11.30.02.20.28; Wed, 30 Nov 2022 02:20:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235185AbiK3J4Y (ORCPT + 84 others); Wed, 30 Nov 2022 04:56:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235099AbiK3J4U (ORCPT ); Wed, 30 Nov 2022 04:56:20 -0500 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C4442E684 for ; Wed, 30 Nov 2022 01:56:13 -0800 (PST) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=hsiangkao@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VW2qBWd_1669802166; Received: from e18g06460.et15sqa.tbsite.net(mailfrom:hsiangkao@linux.alibaba.com fp:SMTPD_---0VW2qBWd_1669802166) by smtp.aliyun-inc.com; Wed, 30 Nov 2022 17:56:11 +0800 From: Gao Xiang To: linux-erofs@lists.ozlabs.org Cc: LKML , Chao Yu , Gao Xiang Subject: [PATCH] erofs: update documentation Date: Wed, 30 Nov 2022 17:56:05 +0800 Message-Id: <20221130095605.4656-1-hsiangkao@linux.alibaba.com> X-Mailer: git-send-email 2.24.4 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org - Refine highlights for main features; - Add multi-reference pclusters and fragment description. Signed-off-by: Gao Xiang --- Documentation/filesystems/erofs.rst | 35 ++++++++++++++++++----------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/Documentation/filesystems/erofs.rst b/Documentation/filesystems/erofs.rst index 05e03d54af1a..82af67fdaf99 100644 --- a/Documentation/filesystems/erofs.rst +++ b/Documentation/filesystems/erofs.rst @@ -30,12 +30,17 @@ It is implemented to be a better choice for the following scenarios: especially for those embedded devices with limited memory and high-density hosts with numerous containers. -Here is the main features of EROFS: +Here are the main features of EROFS: - Little endian on-disk design; - - 4KiB block size and 32-bit block addresses, therefore 16TiB address space - at most for now; + - Block-based and file-based distribution over fscache are supported; + + - Support multiple devices to refer to external blobs, which can be used + for container images; + + - 4KiB block size and 32-bit block addresses for each device, therefore + 16TiB address space at most for now; - Two inode layouts for different requirements: @@ -50,28 +55,29 @@ Here is the main features of EROFS: Metadata reserved 8 bytes 18 bytes ===================== ============ ====================================== - - Metadata and data could be mixed as an option; - - - Support extended attributes (xattrs) as an option; + - Support extended attributes as an option; - - Support tailpacking data and xattr inline compared to byte-addressed - unaligned metadata or smaller block size alternatives; - - - Support POSIX.1e ACLs by using xattrs; + - Support POSIX.1e ACLs by using extended attributes; - Support transparent data compression as an option: LZ4 and MicroLZMA algorithms can be used on a per-file basis; In addition, inplace decompression is also supported to avoid bounce compressed buffers and page cache thrashing. + - Support chunk-based data deduplication and rolling-hash compressed data + deduplication; + + - Support tailpacking inline compared to byte-addressed unaligned metadata + or smaller block size alternatives; + + - Support merging tail-end data into a special inode as fragments. + - Support direct I/O on uncompressed files to avoid double caching for loop devices; - Support FSDAX on uncompressed images for secure containers and ramdisks in order to get rid of unnecessary page cache. - - Support multiple devices for multi blob container images; - - Support file-based on-demand loading with the Fscache infrastructure. The following git tree provides the file system user-space tools under @@ -259,7 +265,7 @@ By the way, chunk-based files are all uncompressed for now. Data compression ---------------- -EROFS implements LZ4 fixed-sized output compression which generates fixed-sized +EROFS implements fixed-sized output compression which generates fixed-sized compressed data blocks from variable-sized input in contrast to other existing fixed-sized input solutions. Relatively higher compression ratios can be gotten by using fixed-sized output compression since nowadays popular data compression @@ -314,3 +320,6 @@ to understand its delta0 is constantly 1, as illustrated below:: If another HEAD follows a HEAD lcluster, there is no room to record CBLKCNT, but it's easy to know the size of such pcluster is 1 lcluster as well. + +Since Linux v6.1, each pcluster can be used for multiple variable-sized extents, +therefore it can be used for compressed data deduplication. -- 2.24.4