Received: by 2002:a05:6902:102b:0:0:0:0 with SMTP id x11csp4320589ybt; Mon, 6 Jul 2020 01:26:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwm8RPShsxdk1pviqVewhjAfTEbPHAehYBXH90wBgArQDxN+6rrgqQm/M0e6DvodOhMpI4H X-Received: by 2002:a17:906:ca56:: with SMTP id jx22mr41628501ejb.494.1594024016050; Mon, 06 Jul 2020 01:26:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594024016; cv=none; d=google.com; s=arc-20160816; b=g/fZCujObJOksvZzKhjJNbbO0Pz+KV0UopkT432YEiJCJPuQT4UBDMf9xyTAjZqQLN p8QMP3aVR79HWKVsm8rWL+/Zb6f6Spwxoet9mzaVfXM9gTgXeJxdM/zp42DGlryKQJ7l Aok/2YEImHnlGz8i+qPifvZjmCvmrXLXy+ZLqOhkF2zCt4e1sMZdS3JKUdAFTMDadXiD hk0FzltdZBD7JwylddGXewWdoC9wJx1gXDa8L5Jk83968+c4PzBOgSTzQE8a45qnO/9b N1jEa+1n54rcB+Bv4cLUSbSdxBGZfTpz2mXCsi30HWWxBHj4ZN1OgBE9n0T1YKB7X5Fh Kuiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject; bh=NdObz4/eAlNw4xrxIyOT+aFxIW2uOyWubgwa1eXBb6E=; b=D104ctqrbdbf0ZQHWJa5l8ERHTfABUiIfMZe3bQ3HF8xNVCHujqY5TQbdR1yguhS20 sk6XuGoLHuNDCgL0RzV2gn9TsubwbUa1Q6ntydbPX3HHl0rblYfY2z2Wlqj61mjuXENn 40oN4pADOMXemrs9PeqM1CX9i7IA4DkWWBASa08kKFEvOGtepxLB5eq7JuB1fmf7L9pE Jm/C9cYWrX3aZt/QPTqklSVtNyBACzgktEg9/m51OcmCp+1eweX36lLEYlrjMjswoDQe 9a04F5jlSiUEglVlmEvWx4NuEa6RPD3fJRR14nz5GebxGGfqSAY5WINLe2lbemWmMmx0 +/dA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id rs25si11949041ejb.536.2020.07.06.01.26.32; Mon, 06 Jul 2020 01:26:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728295AbgGFIZ6 (ORCPT + 99 others); Mon, 6 Jul 2020 04:25:58 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:7373 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726277AbgGFIZ6 (ORCPT ); Mon, 6 Jul 2020 04:25:58 -0400 Received: from DGGEMS409-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 4FA6BC70C4CE2936EBA5; Mon, 6 Jul 2020 16:25:53 +0800 (CST) Received: from [10.134.22.195] (10.134.22.195) by smtp.huawei.com (10.3.19.209) with Microsoft SMTP Server (TLS) id 14.3.487.0; Mon, 6 Jul 2020 16:25:49 +0800 Subject: Re: [PATCH RFC 5/5] f2fs: support age threshold based garbage collection To: CC: , , References: <20200630100428.19105-1-yuchao0@huawei.com> <20200630100428.19105-5-yuchao0@huawei.com> From: Chao Yu Message-ID: <84332ea5-f7b2-f996-31de-b6189c09f27d@huawei.com> Date: Mon, 6 Jul 2020 16:25:48 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20200630100428.19105-5-yuchao0@huawei.com> Content-Type: text/plain; charset="windows-1252" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.134.22.195] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Jaegeuk, This is a commercialized feature in huawei products for years, I think it's time to try to upstream it, could you please take a look at this idea, to see whether it could be a formal feature of f2fs? On 2020/6/30 18:04, Chao Yu wrote: > There are several issues in current background GC algorithm: > - valid blocks is one of key factors during cost overhead calculation, > so if segment has less valid block, however even its age is young or > it locates hot segment, CB algorithm will still choose the segment as > victim, it's not appropriate. > - GCed data/node will go to existing logs, no matter in-there datas' > update frequency is the same or not, it may mix hot and cold data > again. > - GC alloctor mainly use LFS type segment, it will cost free segment > more quickly. > > This patch introduces a new algorithm named age threshold based > garbage collection to solve above issues, there are three steps > mainly: > > 1. select a source victim: > - set an age threshold, and select candidates beased threshold: > e.g. > 0 means youngest, 100 means oldest, if we set age threshold to 80 > then select dirty segments which has age in range of [80, 100] as > candiddates; > - set candidate_ratio threshold, and select candidates based the > ratio, so that we can shrink candidates to those oldest segments; > - select target segment with fewest valid blocks in order to > migrate blocks with minimum cost; > > 2. select a target victim: > - select candidates beased age threshold; > - set candidate_radius threshold, search candidates whose age is > around source victims, searching radius should less than the > radius threshold. > - select target segment with most valid blocks in order to avoid > migrating current target segment. > > 3. merge valid blocks from source victim into target victim with > SSR alloctor. > > Test steps: > - create 160 dirty segments: > * half of them have 128 valid blocks per segment > * left of them have 384 valid blocks per segment > - run background GC > > Benefit: GC count and block movement count both decrease obviously: > > - Before: > - Valid: 86 > - Dirty: 1 > - Prefree: 11 > - Free: 6001 (6001) > > GC calls: 162 (BG: 220) > - data segments : 160 (160) > - node segments : 2 (2) > Try to move 41454 blocks (BG: 41454) > - data blocks : 40960 (40960) > - node blocks : 494 (494) > > IPU: 0 blocks > SSR: 0 blocks in 0 segments > LFS: 41364 blocks in 81 segments > > - After: > > - Valid: 87 > - Dirty: 0 > - Prefree: 4 > - Free: 6008 (6008) > > GC calls: 75 (BG: 76) > - data segments : 74 (74) > - node segments : 1 (1) > Try to move 12813 blocks (BG: 12813) > - data blocks : 12544 (12544) > - node blocks : 269 (269) > > IPU: 0 blocks > SSR: 12032 blocks in 77 segments > LFS: 855 blocks in 2 segments