Received: by 2002:a05:6a10:a0d1:0:0:0:0 with SMTP id j17csp2511682pxa; Mon, 3 Aug 2020 18:50:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxL4YOLznKlXcOucE0TzvLYAVD/4UHrBWL+2QbBhRjSUoHTxT9lwiSzvGlozy17P598DbdR X-Received: by 2002:a50:9987:: with SMTP id m7mr18036692edb.176.1596505839063; Mon, 03 Aug 2020 18:50:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1596505839; cv=none; d=google.com; s=arc-20160816; b=CEf7j7K46NYVXl8CYLxgObqdmgiphjdnrkwhLMKzpXuczDe9XuFx1SKOdfhcn9WIFH rfnNyeH3wWZOp40+kHlYTDguLI3qYhIl/zNe0CrU+e4JcNMLut/Br0rB8P+zVLS5Txr/ qqnyHHALRki06WoLEBdLwyNZyzW4iUd0nglmufD+LO0DH4zWVzpXobXQ0zZ/z8yHNSm3 1it13uQf+QzLMmZL2rAjse7aLRty3GMbEXn4/+dIM1mLF/JHRo47yUobSkWdsGbYJDJR SXxQJcopb6QZvioVQwpS26WiaXOPmKKYN4b9q4/HB5PyGkcPHdRSs8qtGTW3HUUvooWv dMrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:references:cc:to:from:subject; bh=U5lTlp6Ee24Js8mmTG1qC0nWFClnjawiALTTdJ3V9aQ=; b=c5OnJ1Lf1Yp19S92SY509JQ1awKfAdDGKmfUbuggQCO9tYWb98m74k0Y8d/1OCnXx9 m5nGUqXIwHhYEGKXO1GouOkioSZlb5WHj0DrDtZ5oFJRfvURA71/oFxOxI5GFJ+4ugX4 O6b0P7yWCpd8rCdMxFhrzgydAq598rE6lCPggtqNfF+dyZs/OdxyiTcBwUCbdr+Z6qg0 u61vuJtKmPfCmxj5+OEycS3HkS//EKX4KLpOYignizvHSKPDgI39yBt4ifbrA7KYhyZG cO9j2hjSlSXWktP4vVPJubuLAotm0j3asTSFPMigI25HM5Q2r1J3Zrt8BtHj74Xh9bWr zPqQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id dm3si5705894ejc.513.2020.08.03.18.50.16; Mon, 03 Aug 2020 18:50:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728419AbgHDBt1 (ORCPT + 99 others); Mon, 3 Aug 2020 21:49:27 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:9323 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726823AbgHDBt0 (ORCPT ); Mon, 3 Aug 2020 21:49:26 -0400 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id E4722279391636FE1C2E; Tue, 4 Aug 2020 09:49:21 +0800 (CST) Received: from [10.164.122.247] (10.164.122.247) by smtp.huawei.com (10.3.19.206) with Microsoft SMTP Server (TLS) id 14.3.487.0; Tue, 4 Aug 2020 09:49:16 +0800 Subject: Re: [f2fs-dev] [PATCH RFC 1/5] f2fs: introduce inmem curseg From: Chao Yu To: Jaegeuk Kim CC: , References: <20200630100428.19105-1-yuchao0@huawei.com> <20200707032106.GA3139161@google.com> <20200707035142.GB3142230@google.com> <20200715190707.GA2232118@google.com> <976f6a9d-84ad-f269-e210-4326b2ea6a28@huawei.com> <2d336464-47c5-5945-a67b-fb7be8458570@huawei.com> Message-ID: <590ff630-794a-d1de-cde3-da3abca356ed@huawei.com> Date: Tue, 4 Aug 2020 09:49:16 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <2d336464-47c5-5945-a67b-fb7be8458570@huawei.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.164.122.247] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2020/7/25 16:42, Chao Yu wrote: > On 2020/7/16 9:24, Chao Yu wrote: >> On 2020/7/16 3:07, Jaegeuk Kim wrote: >>> On 07/15, Chao Yu wrote: >>>> On 2020/7/7 11:51, Jaegeuk Kim wrote: >>>>> On 07/07, Chao Yu wrote: >>>>>> On 2020/7/7 11:21, Jaegeuk Kim wrote: >>>>>>> Hi Chao, >>>>>>> >>>>>>> Do you have any brief design doc to present the idea? >>>>>> >>>>>> Hi Jaegeuk, >>>>>> >>>>>> You mean this whole patchset, right? >>>>>> >>>>>> I can add a brief design description in patch 0/5. >>>>> >>>>> Yeah, it's a bit hard to understand the whole flow. >>>> >>>> Jaegeuk, >>>> >>>> Do you have time to take a look at this idea summarized in >>>> [PATCH 0/5]'s cover letter? >>> >>> Sorry, I couldn't afford to sitting down to review the design. >>> Let me give it a try soon. >> >> Alright, let me know if you have any question about the idea. > > Ping, > > Jaegeuk, still be too busy...? :P Ping again... Any thoughts about left patches in patchset? > > Thanks, > >> >>> >>>> >>>>> >>>>> Thanks, >>>>> >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> On 06/30, Chao Yu wrote: >>>>>>>> Previous implementation of aligned pinfile allocation will: >>>>>>>> - allocate new segment on cold data log no matter whether last used >>>>>>>> segment is partially used or not, it makes IOs more random; >>>>>>>> - force concurrent cold data/GCed IO going into warm data area, it >>>>>>>> can make a bad effect on hot/cold data separation; >>>>>>>> >>>>>>>> In this patch, we introduce a new type of log named 'inmem curseg', >>>>>>>> the differents from normal curseg is: >>>>>>>> - it reuses existed segment type (CURSEG_XXX_NODE/DATA); >>>>>>>> - it only exists in memory, its segno, blkofs, summary will not b >>>>>>>> persisted into checkpoint area; >>>>>>>> >>>>>>>> With this new feature, we can enhance scalability of log, special >>>>>>>> allocators can be created for purposes: >>>>>>>> - pure lfs allocator for aligned pinfile allocation or file >>>>>>>> defragmentation >>>>>>>> - pure ssr allocator for later feature >>>>>>>> >>>>>>>> So that, let's update aligned pinfile allocation to use this new >>>>>>>> inmem curseg fwk. >>>>>>>> >>>>>>>> Signed-off-by: Chao Yu >>>>>>>> --- >>>>>>>> fs/f2fs/checkpoint.c | 7 ++- >>>>>>>> fs/f2fs/debug.c | 6 ++- >>>>>>>> fs/f2fs/f2fs.h | 12 +++-- >>>>>>>> fs/f2fs/file.c | 3 +- >>>>>>>> fs/f2fs/gc.c | 2 +- >>>>>>>> fs/f2fs/segment.c | 107 ++++++++++++++++++++++++++++++------------- >>>>>>>> fs/f2fs/segment.h | 17 ++++--- >>>>>>>> fs/f2fs/super.c | 9 ++-- >>>>>>>> 8 files changed, 112 insertions(+), 51 deletions(-) >>>>>>>> >>>>>>>> diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c >>>>>>>> index 1bb8278a1c4a..644a914af25a 100644 >>>>>>>> --- a/fs/f2fs/checkpoint.c >>>>>>>> +++ b/fs/f2fs/checkpoint.c >>>>>>>> @@ -1623,11 +1623,16 @@ int f2fs_write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc) >>>>>>>> >>>>>>>> f2fs_flush_sit_entries(sbi, cpc); >>>>>>>> >>>>>>>> + /* save inmem log status */ >>>>>>>> + f2fs_save_inmem_curseg(sbi, CURSEG_COLD_DATA_PINNED); >>>>>>>> + >>>>>>>> err = do_checkpoint(sbi, cpc); >>>>>>>> if (err) >>>>>>>> f2fs_release_discard_addrs(sbi); >>>>>>>> else >>>>>>>> f2fs_clear_prefree_segments(sbi, cpc); >>>>>>>> + >>>>>>>> + f2fs_restore_inmem_curseg(sbi, CURSEG_COLD_DATA_PINNED); >>>>>>>> stop: >>>>>>>> unblock_operations(sbi); >>>>>>>> stat_inc_cp_count(sbi->stat_info); >>>>>>>> @@ -1658,7 +1663,7 @@ void f2fs_init_ino_entry_info(struct f2fs_sb_info *sbi) >>>>>>>> } >>>>>>>> >>>>>>>> sbi->max_orphans = (sbi->blocks_per_seg - F2FS_CP_PACKS - >>>>>>>> - NR_CURSEG_TYPE - __cp_payload(sbi)) * >>>>>>>> + NR_CURSEG_PERSIST_TYPE - __cp_payload(sbi)) * >>>>>>>> F2FS_ORPHANS_PER_BLOCK; >>>>>>>> } >>>>>>>> >>>>>>>> diff --git a/fs/f2fs/debug.c b/fs/f2fs/debug.c >>>>>>>> index 4276c0f79beb..41a91aa8c262 100644 >>>>>>>> --- a/fs/f2fs/debug.c >>>>>>>> +++ b/fs/f2fs/debug.c >>>>>>>> @@ -164,7 +164,7 @@ static void update_general_status(struct f2fs_sb_info *sbi) >>>>>>>> * 100 / (int)(sbi->user_block_count >> sbi->log_blocks_per_seg) >>>>>>>> / 2; >>>>>>>> si->util_invalid = 50 - si->util_free - si->util_valid; >>>>>>>> - for (i = CURSEG_HOT_DATA; i <= CURSEG_COLD_NODE; i++) { >>>>>>>> + for (i = CURSEG_HOT_DATA; i < NO_CHECK_TYPE; i++) { >>>>>>>> struct curseg_info *curseg = CURSEG_I(sbi, i); >>>>>>>> si->curseg[i] = curseg->segno; >>>>>>>> si->cursec[i] = GET_SEC_FROM_SEG(sbi, curseg->segno); >>>>>>>> @@ -393,6 +393,10 @@ static int stat_show(struct seq_file *s, void *v) >>>>>>>> si->dirty_seg[CURSEG_COLD_NODE], >>>>>>>> si->full_seg[CURSEG_COLD_NODE], >>>>>>>> si->valid_blks[CURSEG_COLD_NODE]); >>>>>>>> + seq_printf(s, " - Pinned file: %8d %8d %8d\n", >>>>>>>> + si->curseg[CURSEG_COLD_DATA_PINNED], >>>>>>>> + si->cursec[CURSEG_COLD_DATA_PINNED], >>>>>>>> + si->curzone[CURSEG_COLD_DATA_PINNED]); >>>>>>>> seq_printf(s, "\n - Valid: %d\n - Dirty: %d\n", >>>>>>>> si->main_area_segs - si->dirty_count - >>>>>>>> si->prefree_count - si->free_segs, >>>>>>>> diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h >>>>>>>> index 7d6c5f8ce16b..f06c77066284 100644 >>>>>>>> --- a/fs/f2fs/f2fs.h >>>>>>>> +++ b/fs/f2fs/f2fs.h >>>>>>>> @@ -996,7 +996,9 @@ static inline void set_new_dnode(struct dnode_of_data *dn, struct inode *inode, >>>>>>>> */ >>>>>>>> #define NR_CURSEG_DATA_TYPE (3) >>>>>>>> #define NR_CURSEG_NODE_TYPE (3) >>>>>>>> -#define NR_CURSEG_TYPE (NR_CURSEG_DATA_TYPE + NR_CURSEG_NODE_TYPE) >>>>>>>> +#define NR_CURSEG_INMEM_TYPE (1) >>>>>>>> +#define NR_CURSEG_PERSIST_TYPE (NR_CURSEG_DATA_TYPE + NR_CURSEG_NODE_TYPE) >>>>>>>> +#define NR_CURSEG_TYPE (NR_CURSEG_INMEM_TYPE + NR_CURSEG_PERSIST_TYPE) >>>>>>>> >>>>>>>> enum { >>>>>>>> CURSEG_HOT_DATA = 0, /* directory entry blocks */ >>>>>>>> @@ -1005,8 +1007,10 @@ enum { >>>>>>>> CURSEG_HOT_NODE, /* direct node blocks of directory files */ >>>>>>>> CURSEG_WARM_NODE, /* direct node blocks of normal files */ >>>>>>>> CURSEG_COLD_NODE, /* indirect node blocks */ >>>>>>>> - NO_CHECK_TYPE, >>>>>>>> - CURSEG_COLD_DATA_PINNED,/* cold data for pinned file */ >>>>>>>> + NR_PERSISTENT_LOG, /* number of persistent log */ >>>>>>>> + CURSEG_COLD_DATA_PINNED = NR_PERSISTENT_LOG, >>>>>>>> + /* pinned file that needs consecutive block address */ >>>>>>>> + NO_CHECK_TYPE, /* number of persistent & inmem log */ >>>>>>>> }; >>>>>>>> >>>>>>>> struct flush_cmd { >>>>>>>> @@ -3359,6 +3363,8 @@ block_t f2fs_get_unusable_blocks(struct f2fs_sb_info *sbi); >>>>>>>> int f2fs_disable_cp_again(struct f2fs_sb_info *sbi, block_t unusable); >>>>>>>> void f2fs_release_discard_addrs(struct f2fs_sb_info *sbi); >>>>>>>> int f2fs_npages_for_summary_flush(struct f2fs_sb_info *sbi, bool for_ra); >>>>>>>> +void f2fs_save_inmem_curseg(struct f2fs_sb_info *sbi, int type); >>>>>>>> +void f2fs_restore_inmem_curseg(struct f2fs_sb_info *sbi, int type); >>>>>>>> void f2fs_allocate_segment_for_resize(struct f2fs_sb_info *sbi, int type, >>>>>>>> unsigned int start, unsigned int end); >>>>>>>> void f2fs_allocate_new_segment(struct f2fs_sb_info *sbi, int type); >>>>>>>> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c >>>>>>>> index c10e82806c2a..8611ade06018 100644 >>>>>>>> --- a/fs/f2fs/file.c >>>>>>>> +++ b/fs/f2fs/file.c >>>>>>>> @@ -1656,13 +1656,14 @@ static int expand_inode_data(struct inode *inode, loff_t offset, >>>>>>>> } >>>>>>>> >>>>>>>> down_write(&sbi->pin_sem); >>>>>>>> - map.m_seg_type = CURSEG_COLD_DATA_PINNED; >>>>>>>> >>>>>>>> f2fs_lock_op(sbi); >>>>>>>> f2fs_allocate_new_segment(sbi, CURSEG_COLD_DATA); >>>>>>>> f2fs_unlock_op(sbi); >>>>>>>> >>>>>>>> + map.m_seg_type = CURSEG_COLD_DATA_PINNED; >>>>>>>> err = f2fs_map_blocks(inode, &map, 1, F2FS_GET_BLOCK_PRE_DIO); >>>>>>>> + >>>>>>>> up_write(&sbi->pin_sem); >>>>>>>> >>>>>>>> done += map.m_len; >>>>>>>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c >>>>>>>> index 3b718da69910..84807abe4e00 100644 >>>>>>>> --- a/fs/f2fs/gc.c >>>>>>>> +++ b/fs/f2fs/gc.c >>>>>>>> @@ -1448,7 +1448,7 @@ static int free_segment_range(struct f2fs_sb_info *sbi, >>>>>>>> mutex_unlock(&DIRTY_I(sbi)->seglist_lock); >>>>>>>> >>>>>>>> /* Move out cursegs from the target range */ >>>>>>>> - for (type = CURSEG_HOT_DATA; type < NR_CURSEG_TYPE; type++) >>>>>>>> + for (type = CURSEG_HOT_DATA; type < NR_CURSEG_PERSIST_TYPE; type++) >>>>>>>> f2fs_allocate_segment_for_resize(sbi, type, start, end); >>>>>>>> >>>>>>>> /* do GC to move out valid blocks in the range */ >>>>>>>> diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c >>>>>>>> index 5924b3965ae4..863ec6f1fb87 100644 >>>>>>>> --- a/fs/f2fs/segment.c >>>>>>>> +++ b/fs/f2fs/segment.c >>>>>>>> @@ -1958,7 +1958,7 @@ static void set_prefree_as_free_segments(struct f2fs_sb_info *sbi) >>>>>>>> >>>>>>>> mutex_lock(&dirty_i->seglist_lock); >>>>>>>> for_each_set_bit(segno, dirty_i->dirty_segmap[PRE], MAIN_SEGS(sbi)) >>>>>>>> - __set_test_and_free(sbi, segno); >>>>>>>> + __set_test_and_free(sbi, segno, false); >>>>>>>> mutex_unlock(&dirty_i->seglist_lock); >>>>>>>> } >>>>>>>> >>>>>>>> @@ -2496,6 +2496,7 @@ static void reset_curseg(struct f2fs_sb_info *sbi, int type, int modified) >>>>>>>> struct curseg_info *curseg = CURSEG_I(sbi, type); >>>>>>>> struct summary_footer *sum_footer; >>>>>>>> >>>>>>>> + curseg->inited = true; >>>>>>>> curseg->segno = curseg->next_segno; >>>>>>>> curseg->zone = GET_ZONE_FROM_SEG(sbi, curseg->segno); >>>>>>>> curseg->next_blkoff = 0; >>>>>>>> @@ -2503,24 +2504,31 @@ static void reset_curseg(struct f2fs_sb_info *sbi, int type, int modified) >>>>>>>> >>>>>>>> sum_footer = &(curseg->sum_blk->footer); >>>>>>>> memset(sum_footer, 0, sizeof(struct summary_footer)); >>>>>>>> - if (IS_DATASEG(type)) >>>>>>>> + if (IS_DATASEG(curseg->seg_type)) >>>>>>>> SET_SUM_TYPE(sum_footer, SUM_TYPE_DATA); >>>>>>>> - if (IS_NODESEG(type)) >>>>>>>> + if (IS_NODESEG(curseg->seg_type)) >>>>>>>> SET_SUM_TYPE(sum_footer, SUM_TYPE_NODE); >>>>>>>> - __set_sit_entry_type(sbi, type, curseg->segno, modified); >>>>>>>> + __set_sit_entry_type(sbi, curseg->seg_type, curseg->segno, modified); >>>>>>>> } >>>>>>>> >>>>>>>> static unsigned int __get_next_segno(struct f2fs_sb_info *sbi, int type) >>>>>>>> { >>>>>>>> + struct curseg_info *curseg = CURSEG_I(sbi, type); >>>>>>>> + >>>>>>>> /* if segs_per_sec is large than 1, we need to keep original policy. */ >>>>>>>> if (__is_large_section(sbi)) >>>>>>>> - return CURSEG_I(sbi, type)->segno; >>>>>>>> + return curseg->segno; >>>>>>>> + >>>>>>>> + /* inmem log may not locate on any segment after mount */ >>>>>>>> + if (!curseg->inited) >>>>>>>> + return 0; >>>>>>>> >>>>>>>> if (unlikely(is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >>>>>>>> return 0; >>>>>>>> >>>>>>>> if (test_opt(sbi, NOHEAP) && >>>>>>>> - (type == CURSEG_HOT_DATA || IS_NODESEG(type))) >>>>>>>> + (curseg->seg_type == CURSEG_HOT_DATA || >>>>>>>> + IS_NODESEG(curseg->seg_type))) >>>>>>>> return 0; >>>>>>>> >>>>>>>> if (SIT_I(sbi)->last_victim[ALLOC_NEXT]) >>>>>>>> @@ -2530,7 +2538,7 @@ static unsigned int __get_next_segno(struct f2fs_sb_info *sbi, int type) >>>>>>>> if (F2FS_OPTION(sbi).alloc_mode == ALLOC_MODE_REUSE) >>>>>>>> return 0; >>>>>>>> >>>>>>>> - return CURSEG_I(sbi, type)->segno; >>>>>>>> + return curseg->segno; >>>>>>>> } >>>>>>>> >>>>>>>> /* >>>>>>>> @@ -2540,12 +2548,14 @@ static unsigned int __get_next_segno(struct f2fs_sb_info *sbi, int type) >>>>>>>> static void new_curseg(struct f2fs_sb_info *sbi, int type, bool new_sec) >>>>>>>> { >>>>>>>> struct curseg_info *curseg = CURSEG_I(sbi, type); >>>>>>>> + unsigned short seg_type = curseg->seg_type; >>>>>>>> unsigned int segno = curseg->segno; >>>>>>>> int dir = ALLOC_LEFT; >>>>>>>> >>>>>>>> - write_sum_page(sbi, curseg->sum_blk, >>>>>>>> + if (curseg->inited) >>>>>>>> + write_sum_page(sbi, curseg->sum_blk, >>>>>>>> GET_SUM_BLOCK(sbi, segno)); >>>>>>>> - if (type == CURSEG_WARM_DATA || type == CURSEG_COLD_DATA) >>>>>>>> + if (seg_type == CURSEG_WARM_DATA || seg_type == CURSEG_COLD_DATA) >>>>>>>> dir = ALLOC_RIGHT; >>>>>>>> >>>>>>>> if (test_opt(sbi, NOHEAP)) >>>>>>>> @@ -2622,6 +2632,43 @@ static void change_curseg(struct f2fs_sb_info *sbi, int type) >>>>>>>> f2fs_put_page(sum_page, 1); >>>>>>>> } >>>>>>>> >>>>>>>> +void f2fs_save_inmem_curseg(struct f2fs_sb_info *sbi, int type) >>>>>>>> +{ >>>>>>>> + struct curseg_info *curseg = CURSEG_I(sbi, type); >>>>>>>> + >>>>>>>> + mutex_lock(&curseg->curseg_mutex); >>>>>>>> + if (!curseg->inited) >>>>>>>> + goto out; >>>>>>>> + >>>>>>>> + if (get_valid_blocks(sbi, curseg->segno, false)) { >>>>>>>> + write_sum_page(sbi, curseg->sum_blk, >>>>>>>> + GET_SUM_BLOCK(sbi, curseg->segno)); >>>>>>>> + } else { >>>>>>>> + mutex_lock(&DIRTY_I(sbi)->seglist_lock); >>>>>>>> + __set_test_and_free(sbi, curseg->segno, true); >>>>>>>> + mutex_unlock(&DIRTY_I(sbi)->seglist_lock); >>>>>>>> + } >>>>>>>> +out: >>>>>>>> + mutex_unlock(&curseg->curseg_mutex); >>>>>>>> +} >>>>>>>> + >>>>>>>> +void f2fs_restore_inmem_curseg(struct f2fs_sb_info *sbi, int type) >>>>>>>> +{ >>>>>>>> + struct curseg_info *curseg = CURSEG_I(sbi, type); >>>>>>>> + >>>>>>>> + mutex_lock(&curseg->curseg_mutex); >>>>>>>> + if (!curseg->inited) >>>>>>>> + goto out; >>>>>>>> + if (get_valid_blocks(sbi, curseg->segno, false)) >>>>>>>> + goto out; >>>>>>>> + >>>>>>>> + mutex_lock(&DIRTY_I(sbi)->seglist_lock); >>>>>>>> + __set_test_and_inuse(sbi, curseg->segno); >>>>>>>> + mutex_unlock(&DIRTY_I(sbi)->seglist_lock); >>>>>>>> +out: >>>>>>>> + mutex_unlock(&curseg->curseg_mutex); >>>>>>>> +} >>>>>>>> + >>>>>>>> static int get_ssr_segment(struct f2fs_sb_info *sbi, int type) >>>>>>>> { >>>>>>>> struct curseg_info *curseg = CURSEG_I(sbi, type); >>>>>>>> @@ -2738,11 +2785,15 @@ void __allocate_new_segment(struct f2fs_sb_info *sbi, int type) >>>>>>>> struct curseg_info *curseg = CURSEG_I(sbi, type); >>>>>>>> unsigned int old_segno; >>>>>>>> >>>>>>>> + if (!curseg->inited) >>>>>>>> + goto alloc; >>>>>>>> + >>>>>>>> if (!curseg->next_blkoff && >>>>>>>> !get_valid_blocks(sbi, curseg->segno, false) && >>>>>>>> !get_ckpt_valid_blocks(sbi, curseg->segno)) >>>>>>>> return; >>>>>>>> >>>>>>>> +alloc: >>>>>>>> old_segno = curseg->segno; >>>>>>>> SIT_I(sbi)->s_ops->allocate_segment(sbi, type, true); >>>>>>>> locate_dirty_segment(sbi, old_segno); >>>>>>>> @@ -3126,19 +3177,6 @@ void f2fs_allocate_data_block(struct f2fs_sb_info *sbi, struct page *page, >>>>>>>> { >>>>>>>> struct sit_info *sit_i = SIT_I(sbi); >>>>>>>> struct curseg_info *curseg = CURSEG_I(sbi, type); >>>>>>>> - bool put_pin_sem = false; >>>>>>>> - >>>>>>>> - if (type == CURSEG_COLD_DATA) { >>>>>>>> - /* GC during CURSEG_COLD_DATA_PINNED allocation */ >>>>>>>> - if (down_read_trylock(&sbi->pin_sem)) { >>>>>>>> - put_pin_sem = true; >>>>>>>> - } else { >>>>>>>> - type = CURSEG_WARM_DATA; >>>>>>>> - curseg = CURSEG_I(sbi, type); >>>>>>>> - } >>>>>>>> - } else if (type == CURSEG_COLD_DATA_PINNED) { >>>>>>>> - type = CURSEG_COLD_DATA; >>>>>>>> - } >>>>>>>> >>>>>>>> down_read(&SM_I(sbi)->curseg_lock); >>>>>>>> >>>>>>>> @@ -3204,9 +3242,6 @@ void f2fs_allocate_data_block(struct f2fs_sb_info *sbi, struct page *page, >>>>>>>> mutex_unlock(&curseg->curseg_mutex); >>>>>>>> >>>>>>>> up_read(&SM_I(sbi)->curseg_lock); >>>>>>>> - >>>>>>>> - if (put_pin_sem) >>>>>>>> - up_read(&sbi->pin_sem); >>>>>>>> } >>>>>>>> >>>>>>>> static void update_device_state(struct f2fs_io_info *fio) >>>>>>>> @@ -3574,7 +3609,7 @@ static int read_normal_summaries(struct f2fs_sb_info *sbi, int type) >>>>>>>> blk_off = le16_to_cpu(ckpt->cur_data_blkoff[type - >>>>>>>> CURSEG_HOT_DATA]); >>>>>>>> if (__exist_node_summaries(sbi)) >>>>>>>> - blk_addr = sum_blk_addr(sbi, NR_CURSEG_TYPE, type); >>>>>>>> + blk_addr = sum_blk_addr(sbi, NR_CURSEG_PERSIST_TYPE, type); >>>>>>>> else >>>>>>>> blk_addr = sum_blk_addr(sbi, NR_CURSEG_DATA_TYPE, type); >>>>>>>> } else { >>>>>>>> @@ -3652,8 +3687,9 @@ static int restore_curseg_summaries(struct f2fs_sb_info *sbi) >>>>>>>> } >>>>>>>> >>>>>>>> if (__exist_node_summaries(sbi)) >>>>>>>> - f2fs_ra_meta_pages(sbi, sum_blk_addr(sbi, NR_CURSEG_TYPE, type), >>>>>>>> - NR_CURSEG_TYPE - type, META_CP, true); >>>>>>>> + f2fs_ra_meta_pages(sbi, >>>>>>>> + sum_blk_addr(sbi, NR_CURSEG_PERSIST_TYPE, type), >>>>>>>> + NR_CURSEG_PERSIST_TYPE - type, META_CP, true); >>>>>>>> >>>>>>>> for (; type <= CURSEG_COLD_NODE; type++) { >>>>>>>> err = read_normal_summaries(sbi, type); >>>>>>>> @@ -4155,14 +4191,14 @@ static int build_curseg(struct f2fs_sb_info *sbi) >>>>>>>> struct curseg_info *array; >>>>>>>> int i; >>>>>>>> >>>>>>>> - array = f2fs_kzalloc(sbi, array_size(NR_CURSEG_TYPE, sizeof(*array)), >>>>>>>> - GFP_KERNEL); >>>>>>>> + array = f2fs_kzalloc(sbi, array_size(NR_CURSEG_TYPE, >>>>>>>> + sizeof(*array)), GFP_KERNEL); >>>>>>>> if (!array) >>>>>>>> return -ENOMEM; >>>>>>>> >>>>>>>> SM_I(sbi)->curseg_array = array; >>>>>>>> >>>>>>>> - for (i = 0; i < NR_CURSEG_TYPE; i++) { >>>>>>>> + for (i = 0; i < NO_CHECK_TYPE; i++) { >>>>>>>> mutex_init(&array[i].curseg_mutex); >>>>>>>> array[i].sum_blk = f2fs_kzalloc(sbi, PAGE_SIZE, GFP_KERNEL); >>>>>>>> if (!array[i].sum_blk) >>>>>>>> @@ -4172,8 +4208,13 @@ static int build_curseg(struct f2fs_sb_info *sbi) >>>>>>>> sizeof(struct f2fs_journal), GFP_KERNEL); >>>>>>>> if (!array[i].journal) >>>>>>>> return -ENOMEM; >>>>>>>> + if (i < NR_PERSISTENT_LOG) >>>>>>>> + array[i].seg_type = CURSEG_HOT_DATA + i; >>>>>>>> + else if (i == CURSEG_COLD_DATA_PINNED) >>>>>>>> + array[i].seg_type = CURSEG_COLD_DATA; >>>>>>>> array[i].segno = NULL_SEGNO; >>>>>>>> array[i].next_blkoff = 0; >>>>>>>> + array[i].inited = false; >>>>>>>> } >>>>>>>> return restore_curseg_summaries(sbi); >>>>>>>> } >>>>>>>> @@ -4408,7 +4449,7 @@ static int sanity_check_curseg(struct f2fs_sb_info *sbi) >>>>>>>> * In LFS/SSR curseg, .next_blkoff should point to an unused blkaddr; >>>>>>>> * In LFS curseg, all blkaddr after .next_blkoff should be unused. >>>>>>>> */ >>>>>>>> - for (i = 0; i < NO_CHECK_TYPE; i++) { >>>>>>>> + for (i = 0; i < NR_PERSISTENT_LOG; i++) { >>>>>>>> struct curseg_info *curseg = CURSEG_I(sbi, i); >>>>>>>> struct seg_entry *se = get_seg_entry(sbi, curseg->segno); >>>>>>>> unsigned int blkofs = curseg->next_blkoff; >>>>>>>> @@ -4637,7 +4678,7 @@ int f2fs_fix_curseg_write_pointer(struct f2fs_sb_info *sbi) >>>>>>>> { >>>>>>>> int i, ret; >>>>>>>> >>>>>>>> - for (i = 0; i < NO_CHECK_TYPE; i++) { >>>>>>>> + for (i = 0; i < NR_PERSISTENT_LOG; i++) { >>>>>>>> ret = fix_curseg_write_pointer(sbi, i); >>>>>>>> if (ret) >>>>>>>> return ret; >>>>>>>> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >>>>>>>> index f261e3e6a69b..8ff261550cbb 100644 >>>>>>>> --- a/fs/f2fs/segment.h >>>>>>>> +++ b/fs/f2fs/segment.h >>>>>>>> @@ -22,7 +22,7 @@ >>>>>>>> #define GET_R2L_SEGNO(free_i, segno) ((segno) + (free_i)->start_segno) >>>>>>>> >>>>>>>> #define IS_DATASEG(t) ((t) <= CURSEG_COLD_DATA) >>>>>>>> -#define IS_NODESEG(t) ((t) >= CURSEG_HOT_NODE) >>>>>>>> +#define IS_NODESEG(t) ((t) >= CURSEG_HOT_NODE && (t) <= CURSEG_COLD_NODE) >>>>>>>> >>>>>>>> #define IS_HOT(t) ((t) == CURSEG_HOT_NODE || (t) == CURSEG_HOT_DATA) >>>>>>>> #define IS_WARM(t) ((t) == CURSEG_WARM_NODE || (t) == CURSEG_WARM_DATA) >>>>>>>> @@ -34,7 +34,8 @@ >>>>>>>> ((seg) == CURSEG_I(sbi, CURSEG_COLD_DATA)->segno) || \ >>>>>>>> ((seg) == CURSEG_I(sbi, CURSEG_HOT_NODE)->segno) || \ >>>>>>>> ((seg) == CURSEG_I(sbi, CURSEG_WARM_NODE)->segno) || \ >>>>>>>> - ((seg) == CURSEG_I(sbi, CURSEG_COLD_NODE)->segno)) >>>>>>>> + ((seg) == CURSEG_I(sbi, CURSEG_COLD_NODE)->segno) || \ >>>>>>>> + ((seg) == CURSEG_I(sbi, CURSEG_COLD_DATA_PINNED)->segno)) >>>>>>>> >>>>>>>> #define IS_CURSEC(sbi, secno) \ >>>>>>>> (((secno) == CURSEG_I(sbi, CURSEG_HOT_DATA)->segno / \ >>>>>>>> @@ -48,7 +49,9 @@ >>>>>>>> ((secno) == CURSEG_I(sbi, CURSEG_WARM_NODE)->segno / \ >>>>>>>> (sbi)->segs_per_sec) || \ >>>>>>>> ((secno) == CURSEG_I(sbi, CURSEG_COLD_NODE)->segno / \ >>>>>>>> - (sbi)->segs_per_sec)) \ >>>>>>>> + (sbi)->segs_per_sec) || \ >>>>>>>> + ((secno) == CURSEG_I(sbi, CURSEG_COLD_DATA_PINNED)->segno / \ >>>>>>>> + (sbi)->segs_per_sec)) >>>>>>>> >>>>>>>> #define MAIN_BLKADDR(sbi) \ >>>>>>>> (SM_I(sbi) ? SM_I(sbi)->main_blkaddr : \ >>>>>>>> @@ -288,10 +291,12 @@ struct curseg_info { >>>>>>>> struct rw_semaphore journal_rwsem; /* protect journal area */ >>>>>>>> struct f2fs_journal *journal; /* cached journal info */ >>>>>>>> unsigned char alloc_type; /* current allocation type */ >>>>>>>> + unsigned short seg_type; /* segment type like CURSEG_XXX_TYPE */ >>>>>>>> unsigned int segno; /* current segment number */ >>>>>>>> unsigned short next_blkoff; /* next block offset to write */ >>>>>>>> unsigned int zone; /* current zone number */ >>>>>>>> unsigned int next_segno; /* preallocated segment */ >>>>>>>> + bool inited; /* indicate inmem log is inited */ >>>>>>>> }; >>>>>>>> >>>>>>>> struct sit_entry_set { >>>>>>>> @@ -305,8 +310,6 @@ struct sit_entry_set { >>>>>>>> */ >>>>>>>> static inline struct curseg_info *CURSEG_I(struct f2fs_sb_info *sbi, int type) >>>>>>>> { >>>>>>>> - if (type == CURSEG_COLD_DATA_PINNED) >>>>>>>> - type = CURSEG_COLD_DATA; >>>>>>>> return (struct curseg_info *)(SM_I(sbi)->curseg_array + type); >>>>>>>> } >>>>>>>> >>>>>>>> @@ -438,7 +441,7 @@ static inline void __set_inuse(struct f2fs_sb_info *sbi, >>>>>>>> } >>>>>>>> >>>>>>>> static inline void __set_test_and_free(struct f2fs_sb_info *sbi, >>>>>>>> - unsigned int segno) >>>>>>>> + unsigned int segno, bool inmem) >>>>>>>> { >>>>>>>> struct free_segmap_info *free_i = FREE_I(sbi); >>>>>>>> unsigned int secno = GET_SEC_FROM_SEG(sbi, segno); >>>>>>>> @@ -449,7 +452,7 @@ static inline void __set_test_and_free(struct f2fs_sb_info *sbi, >>>>>>>> if (test_and_clear_bit(segno, free_i->free_segmap)) { >>>>>>>> free_i->free_segments++; >>>>>>>> >>>>>>>> - if (IS_CURSEC(sbi, secno)) >>>>>>>> + if (!inmem && IS_CURSEC(sbi, secno)) >>>>>>>> goto skip_free; >>>>>>>> next = find_next_bit(free_i->free_segmap, >>>>>>>> start_segno + sbi->segs_per_sec, start_segno); >>>>>>>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c >>>>>>>> index 80cb7cd358f8..0fefa130585f 100644 >>>>>>>> --- a/fs/f2fs/super.c >>>>>>>> +++ b/fs/f2fs/super.c >>>>>>>> @@ -575,7 +575,8 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount) >>>>>>>> case Opt_active_logs: >>>>>>>> if (args->from && match_int(args, &arg)) >>>>>>>> return -EINVAL; >>>>>>>> - if (arg != 2 && arg != 4 && arg != NR_CURSEG_TYPE) >>>>>>>> + if (arg != 2 && arg != 4 && >>>>>>>> + arg != NR_CURSEG_PERSIST_TYPE) >>>>>>>> return -EINVAL; >>>>>>>> F2FS_OPTION(sbi).active_logs = arg; >>>>>>>> break; >>>>>>>> @@ -981,7 +982,7 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount) >>>>>>>> } >>>>>>>> >>>>>>>> /* Not pass down write hints if the number of active logs is lesser >>>>>>>> - * than NR_CURSEG_TYPE. >>>>>>>> + * than NR_CURSEG_PERSIST_TYPE. >>>>>>>> */ >>>>>>>> if (F2FS_OPTION(sbi).active_logs != NR_CURSEG_TYPE) >>>>>>>> F2FS_OPTION(sbi).whint_mode = WHINT_MODE_OFF; >>>>>>>> @@ -1614,7 +1615,7 @@ static int f2fs_show_options(struct seq_file *seq, struct dentry *root) >>>>>>>> static void default_options(struct f2fs_sb_info *sbi) >>>>>>>> { >>>>>>>> /* init some FS parameters */ >>>>>>>> - F2FS_OPTION(sbi).active_logs = NR_CURSEG_TYPE; >>>>>>>> + F2FS_OPTION(sbi).active_logs = NR_CURSEG_PERSIST_TYPE; >>>>>>>> F2FS_OPTION(sbi).inline_xattr_size = DEFAULT_INLINE_XATTR_ADDRS; >>>>>>>> F2FS_OPTION(sbi).whint_mode = WHINT_MODE_OFF; >>>>>>>> F2FS_OPTION(sbi).alloc_mode = ALLOC_MODE_DEFAULT; >>>>>>>> @@ -2946,7 +2947,7 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi) >>>>>>>> cp_payload = __cp_payload(sbi); >>>>>>>> if (cp_pack_start_sum < cp_payload + 1 || >>>>>>>> cp_pack_start_sum > blocks_per_seg - 1 - >>>>>>>> - NR_CURSEG_TYPE) { >>>>>>>> + NR_CURSEG_PERSIST_TYPE) { >>>>>>>> f2fs_err(sbi, "Wrong cp_pack_start_sum: %u", >>>>>>>> cp_pack_start_sum); >>>>>>>> return 1; >>>>>>>> -- >>>>>>>> 2.26.2 >>>>>>> . >>>>>>> >>>>> . >>>>> >>> . >>> >> >> >> _______________________________________________ >> Linux-f2fs-devel mailing list >> Linux-f2fs-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel >> . >> > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > . >