Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <3801a538-f059-2b62-6dbb-cbe67478371b@kernel.org>
Date:   Thu, 21 Jul 2022 22:31:53 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH v2 00/16] erofs: prepare for folios, deduplication and
 kill PG_error
Content-Language: en-US
To:     Gao Xiang <hsiangkao@linux.alibaba.com>,
        linux-erofs@lists.ozlabs.org
Cc:     LKML <linux-kernel@vger.kernel.org>
References: <20220715154203.48093-1-hsiangkao@linux.alibaba.com>
From:   Chao Yu <chao@kernel.org>
In-Reply-To: <20220715154203.48093-1-hsiangkao@linux.alibaba.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2022/7/15 23:41, Gao Xiang wrote:
> Hi folks,
> 
> I've been doing this for almost 2 months, the main point of this is
> to support large folios and rolling hash deduplication for compressed
> data.
> 
> This patchset is as a start of this work targeting for the next 5.20,
> it introduces a flexable range representation for (de)compressed buffers
> instead of too relying on page(s) directly themselves, so large folios
> can laterly base on this work.  Also, this patchset gets rid of all
> PG_error flags in the decompression code. It's a cleanup as a result
> as well.
> 
> In addition, this patchset kicks off rolling hash deduplication for
> compressed data by introducing fully-referenced multi-reference
> pclusters first instead of reporting fs corruption if one pcluster
> is introduced by several differnt extents.  The full implementation
> is expected to be finished in the merge window after the next.  One
> of my colleagues is actively working on the userspace part of this
> feature.
> 
> However, it's still easy to verify fully-referenced multi-reference
> pcluster by constructing some image by hand (see attachment):
> 
> Dataset: 300M
> seq-read (data-deduplicated, read_ahead_kb 8192): 1095MiB/s
> seq-read (data-deduplicated, read_ahead_kb 4096): 771MiB/s
> seq-read (data-deduplicated, read_ahead_kb 512):  577MiB/s
> seq-read (vanilla, read_ahead_kb 8192):           364MiB/s
> 
> Finally, this patchset survives ro-fsstress on my side.

For this patchset,

Acked-by: Chao Yu <chao@kernel.org>

Thanks,

> 
> Thanks,
> Gao Xiang
> 
> Changes since v1:
>   - rename left pagevec words to bvpage (Yue Hu);
> 
> Gao Xiang (16):
>    erofs: get rid of unneeded `inode', `map' and `sb'
>    erofs: clean up z_erofs_collector_begin()
>    erofs: introduce `z_erofs_parse_out_bvecs()'
>    erofs: introduce bufvec to store decompressed buffers
>    erofs: drop the old pagevec approach
>    erofs: introduce `z_erofs_parse_in_bvecs'
>    erofs: switch compressed_pages[] to bufvec
>    erofs: rework online page handling
>    erofs: get rid of `enum z_erofs_page_type'
>    erofs: clean up `enum z_erofs_collectmode'
>    erofs: get rid of `z_pagemap_global'
>    erofs: introduce struct z_erofs_decompress_backend
>    erofs: try to leave (de)compressed_pages on stack if possible
>    erofs: introduce z_erofs_do_decompressed_bvec()
>    erofs: record the longest decompressed size in this round
>    erofs: introduce multi-reference pclusters (fully-referenced)
> 
>   fs/erofs/compress.h     |   2 +-
>   fs/erofs/decompressor.c |   2 +-
>   fs/erofs/zdata.c        | 785 +++++++++++++++++++++++-----------------
>   fs/erofs/zdata.h        | 119 +++---
>   fs/erofs/zpvec.h        | 159 --------
>   5 files changed, 496 insertions(+), 571 deletions(-)
>   delete mode 100644 fs/erofs/zpvec.h
>