- Refine highlights for main features;
- Add multi-reference pclusters and fragment description.
Signed-off-by: Gao Xiang <[email protected]>
---
Documentation/filesystems/erofs.rst | 35 ++++++++++++++++++-----------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/Documentation/filesystems/erofs.rst b/Documentation/filesystems/erofs.rst
index 05e03d54af1a..82af67fdaf99 100644
--- a/Documentation/filesystems/erofs.rst
+++ b/Documentation/filesystems/erofs.rst
@@ -30,12 +30,17 @@ It is implemented to be a better choice for the following scenarios:
especially for those embedded devices with limited memory and high-density
hosts with numerous containers.
-Here is the main features of EROFS:
+Here are the main features of EROFS:
- Little endian on-disk design;
- - 4KiB block size and 32-bit block addresses, therefore 16TiB address space
- at most for now;
+ - Block-based and file-based distribution over fscache are supported;
+
+ - Support multiple devices to refer to external blobs, which can be used
+ for container images;
+
+ - 4KiB block size and 32-bit block addresses for each device, therefore
+ 16TiB address space at most for now;
- Two inode layouts for different requirements:
@@ -50,28 +55,29 @@ Here is the main features of EROFS:
Metadata reserved 8 bytes 18 bytes
===================== ============ ======================================
- - Metadata and data could be mixed as an option;
-
- - Support extended attributes (xattrs) as an option;
+ - Support extended attributes as an option;
- - Support tailpacking data and xattr inline compared to byte-addressed
- unaligned metadata or smaller block size alternatives;
-
- - Support POSIX.1e ACLs by using xattrs;
+ - Support POSIX.1e ACLs by using extended attributes;
- Support transparent data compression as an option:
LZ4 and MicroLZMA algorithms can be used on a per-file basis; In addition,
inplace decompression is also supported to avoid bounce compressed buffers
and page cache thrashing.
+ - Support chunk-based data deduplication and rolling-hash compressed data
+ deduplication;
+
+ - Support tailpacking inline compared to byte-addressed unaligned metadata
+ or smaller block size alternatives;
+
+ - Support merging tail-end data into a special inode as fragments.
+
- Support direct I/O on uncompressed files to avoid double caching for loop
devices;
- Support FSDAX on uncompressed images for secure containers and ramdisks in
order to get rid of unnecessary page cache.
- - Support multiple devices for multi blob container images;
-
- Support file-based on-demand loading with the Fscache infrastructure.
The following git tree provides the file system user-space tools under
@@ -259,7 +265,7 @@ By the way, chunk-based files are all uncompressed for now.
Data compression
----------------
-EROFS implements LZ4 fixed-sized output compression which generates fixed-sized
+EROFS implements fixed-sized output compression which generates fixed-sized
compressed data blocks from variable-sized input in contrast to other existing
fixed-sized input solutions. Relatively higher compression ratios can be gotten
by using fixed-sized output compression since nowadays popular data compression
@@ -314,3 +320,6 @@ to understand its delta0 is constantly 1, as illustrated below::
If another HEAD follows a HEAD lcluster, there is no room to record CBLKCNT,
but it's easy to know the size of such pcluster is 1 lcluster as well.
+
+Since Linux v6.1, each pcluster can be used for multiple variable-sized extents,
+therefore it can be used for compressed data deduplication.
--
2.24.4
On Wed, 30 Nov 2022 17:56:05 +0800
Gao Xiang <[email protected]> wrote:
> - Refine highlights for main features;
>
> - Add multi-reference pclusters and fragment description.
>
> Signed-off-by: Gao Xiang <[email protected]>
Reviewed-by: Yue Hu <[email protected]>
> ---
> Documentation/filesystems/erofs.rst | 35 ++++++++++++++++++-----------
> 1 file changed, 22 insertions(+), 13 deletions(-)
>
On 2022/11/30 17:56, Gao Xiang wrote:
> - Refine highlights for main features;
>
> - Add multi-reference pclusters and fragment description.
>
> Signed-off-by: Gao Xiang <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Thanks,
On Thu, Sep 28, 2023 at 09:16:00PM +0800, Jingbo Xu wrote:
> - update new features like bloom filter and DEFLATE.
>
> - add documentation for the long xattr name prefixes, which was
> landed upstream since v6.4.
>
> Signed-off-by: Jingbo Xu <[email protected]>
> ---
> Documentation/filesystems/erofs.rst | 40 ++++++++++++++++++++++++++---
> 1 file changed, 37 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/filesystems/erofs.rst b/Documentation/filesystems/erofs.rst
> index 4654ee57c1d5..522183737be6 100644
> --- a/Documentation/filesystems/erofs.rst
> +++ b/Documentation/filesystems/erofs.rst
> @@ -58,12 +58,14 @@ Here are the main features of EROFS:
>
> - Support extended attributes as an option;
>
> + - Support a bloom filter that speeds up negative extended attribute lookups;
> +
> - Support POSIX.1e ACLs by using extended attributes;
>
> - Support transparent data compression as an option:
> - LZ4 and MicroLZMA algorithms can be used on a per-file basis; In addition,
> - inplace decompression is also supported to avoid bounce compressed buffers
> - and page cache thrashing.
> + LZ4, MicroLZMA and DEFLATE algorithms can be used on a per-file basis; In
> + addition, inplace decompression is also supported to avoid bounce compressed
> + buffers and unnecessary page cache thrashing.
>
> - Support chunk-based data deduplication and rolling-hash compressed data
> deduplication;
> @@ -268,6 +270,38 @@ details.)
>
> By the way, chunk-based files are all uncompressed for now.
>
> +Long extended attribute name prefixes
> +-------------------------------------
> +There are use cases that extended attributes with different values can have
^ where
> +only a few common prefixes (such as overlayfs xattrs). The predefined prefixes
> +works inefficiently in both image size and runtime performance in such cases.
^ work
> +
> +The long xattr name prefixes feature is introduced to address this issue. The
> +overall idea is that, apart from the existing predefined prefixes, the xattr
> +entry could also refer to user specified long xattr name prefixes, e.g.
^ user-specified
> +"trusted.overlay.".
> +
> +When referring to a long xattr name prefix, the highest bit (bit 7) of
> +erofs_xattr_entry.e_name_index is set, while the lower bits (bit 0-6) as a whole
> +represents the index of the referred long name prefix among all long name
> +prefixes. Therefore, only the trailing part of the name apart from the long
> +xattr name prefix is stored in erofs_xattr_entry.e_name, which could be empty if
> +the full xattr name matches exactly as its long xattr name prefix.
> +
> +All long xattr prefixes are stored one by one in the packed inode as long as
> +the packed inode is valid, or meta inode otherwise. The xattr_prefix_count (of
^ the meta inode
> +on-disk superblock) indicates the total number of the long xattr name prefixes,
^ the on-disk superblock
> +while (xattr_prefix_start * 4) indicates the start offset of long name prefixes
> +in the packed/meta inode. Note that, long extended attribute name prefixes is
^ are
> +disabled if xattr_prefix_count is 0.
> +
> +Each long name prefix is stored in the format: ALIGN({__le16 len, data}, 4),
> +where len represents the total size of the data part. The data part is actually
> +represented by 'struct erofs_xattr_long_prefix', where base_index represents the
> +index of the predefined xattr name prefix, e.g. EROFS_XATTR_INDEX_TRUSTED for
> +"trusted.overlay." long name prefix, while the infix string kepts the string
^ keeps
Thanks,
Gao Xiang