Received: by 2002:ab2:620c:0:b0:1ef:ffd0:ce49 with SMTP id o12csp372478lqt; Mon, 18 Mar 2024 10:07:57 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCV2wJ/q8K8uz1eVw4sRUDqkEiLNciH056YVwAGqy8f8Tez0FBcxNUTXKAC/Og/aQMetm+W4PJTXwQqpv/3thDIyIjskPB7zbS6tXtFBpA== X-Google-Smtp-Source: AGHT+IEZcOb9PhQ8YPLbSfc8DselLPz3jr1vkjrDjbN60zDdZHSYgdWLUsG/mhLIkiatWRAM6W1p X-Received: by 2002:a05:622a:14:b0:430:b470:863d with SMTP id x20-20020a05622a001400b00430b470863dmr10752227qtw.55.1710781677675; Mon, 18 Mar 2024 10:07:57 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1710781677; cv=pass; d=google.com; s=arc-20160816; b=df1mUL9NaGd2U8hmKN4A2c1YaeSFnKFfCRrQURHAJnCFDM5Sl2KwfRMBxerCf3lkod Yo9U6rNMcjfAeiRcyrYBeeGO3lTwMYhpXGpb/0pR6c+JvNqBVZCJGMqqclnrJk/6WiOg vhoh4b8kF+MwboPP7mNjg59fKH5/zmRCETZo2LqJyOwWpxJr0Vll3uFMcrQIDaeHjmCc UvsYQQfeWBZdzM6UYhQaf6iddcBSzc1QCzp2/vSXw4okSjGTQJLFBc8KdA49TapY1BKK FYzW/0laeqc0ajYHHtc4853oPPbEkmxh2Trgna7oS933S2Ln0fbbZjogZcHQUTQym3bK yPxQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:message-id:subject:cc :to:from:date:dkim-signature:dkim-signature:dkim-signature :dkim-signature; bh=qJZVd+/t6/T/I9QFCHsi+gcMomJFVa7ZLsqdOX6uWZg=; fh=ZcSpSL/mjV28IKVsJOlacjbuY4W5ExXAJAgt440gLUE=; b=Qt2Mpw85tJE+TcPgEi9WDOYHZ6SZUI18H736QJaeb32rJJDrZh4A4s3wyAhrtkFIoM 1dxsmQuNJ2cMkOr672PSSRTjyt+8Y/ZDfW2oPEPfzfxbIqGM2ffxJRzDZ4lA3ZbFhQwt tq/VriCguU/yg6TNGhnvj7iZbmpNX7ykD6mzpi8MIHsPrD5b0wne3NoSGmfckIk5952T bc4TmJWESzqHRwnCnVfRWwImBhVyRtpPGl12LF/3CeJqMlAdIGRt2tEssjf+PmT3fWmY XkyvkKgd5njJ6GgrX3DjJ2YQYrZ9Nb4ksvasSNpp0QBOFCVRe81kyCqPsUEH7sBHAV7h UMlQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.cz header.s=susede2_rsa header.b=iXb1PKuu; dkim=neutral (no key) header.i=@suse.cz header.s=susede2_ed25519; dkim=pass header.i=@suse.cz header.s=susede2_rsa header.b=eWbbtdtO; dkim=neutral (no key) header.i=@suse.cz header.s=susede2_ed25519; arc=pass (i=1 spf=pass spfdomain=suse.cz dkim=pass dkdomain=suse.cz dkim=pass dkdomain=suse.cz); spf=pass (google.com: domain of linux-kernel+bounces-106419-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-106419-linux.lists.archive=gmail.com@vger.kernel.org" Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id c18-20020ac87d92000000b0042ef314bd79si10023852qtd.180.2024.03.18.10.07.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 10:07:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-106419-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.cz header.s=susede2_rsa header.b=iXb1PKuu; dkim=neutral (no key) header.i=@suse.cz header.s=susede2_ed25519; dkim=pass header.i=@suse.cz header.s=susede2_rsa header.b=eWbbtdtO; dkim=neutral (no key) header.i=@suse.cz header.s=susede2_ed25519; arc=pass (i=1 spf=pass spfdomain=suse.cz dkim=pass dkdomain=suse.cz dkim=pass dkdomain=suse.cz); spf=pass (google.com: domain of linux-kernel+bounces-106419-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-106419-linux.lists.archive=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 7DF2D1C21114 for ; Mon, 18 Mar 2024 17:07:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7624954FB6; Mon, 18 Mar 2024 17:07:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="iXb1PKuu"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="ArCWwG1v"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="eWbbtdtO"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="SjSCGtwr" Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBE9A3A8F9; Mon, 18 Mar 2024 17:07:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710781651; cv=none; b=H3jgvkDTJN9zd+sEWqJ6q3CTQl1l7V2Lrrdl1ox/HdHGf1J1xDC0UPuQs/DuE9t02yWcihUxF/1i/5PqoPIVrychd3jFj0aNlY0qT57fUpUOaNRhNjIEXJfgBsE/wXSrGtYL4XekJQ4ygA8BCjkAcS39xWg97/KumcudSIKvjes= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710781651; c=relaxed/simple; bh=HUsRDo7Jc5bEEXI3vYu8ImWAopUkQ+k9hXHhlIthcTU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=AY2lS5wc/veQi5dbaYSQXapbXh9fZ0WBL7EqVqwu0Ma38j4bUtspfJCdLSGHd7Sy+LflU99VE5agF4/Y3iAwslLdkRDweb2xUlaRmVbmwDpBlqqvWchw//JtBl7Ef+rNkiFRPoCR/ZAIxWyD5y3zzGh3bzsgifqFTVB8MArc2z4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=iXb1PKuu; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=ArCWwG1v; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=eWbbtdtO; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=SjSCGtwr; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0CC8A5C7BB; Mon, 18 Mar 2024 17:07:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1710781647; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qJZVd+/t6/T/I9QFCHsi+gcMomJFVa7ZLsqdOX6uWZg=; b=iXb1PKuuDGGCUjN++LNeztrlwGPPl5GVIL7V3Tdp+zZdGYaYdahKg+NHbPHbhi/PKzpe7Y dgP8XvlcUGXPWShxVmkN1XpQX34Ux5Vf/Pjo0SLK+aEu5mWoS3BAPAWoFA634S+3eO24eX xucZej1YO5JdNJji+1F0Ev2V16XlszY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1710781647; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qJZVd+/t6/T/I9QFCHsi+gcMomJFVa7ZLsqdOX6uWZg=; b=ArCWwG1vdwHamxfg99VxdZN8c+1mIzBU44jSfNkbuHWVCJJJItjVVkSZANQvxQgWvb78/q aamt7wMT193xJfDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1710781645; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qJZVd+/t6/T/I9QFCHsi+gcMomJFVa7ZLsqdOX6uWZg=; b=eWbbtdtOfLsZiZcNne86jekbsLCwozzSBlkEHIlQrYhWPqGSjo9b+npLAPfX8DkjXFM56t MHz+4VEs3F4lPy22g79/522qIchYPlnaEXahsJThkKrpKJx0dnGCMxfr025QhpLVv0NfYv rAhgGCvxP4nM5Mr/v5ZWUVnTLx9L/tY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1710781645; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qJZVd+/t6/T/I9QFCHsi+gcMomJFVa7ZLsqdOX6uWZg=; b=SjSCGtwrz2xaeId9fVFnKt9Q87nZpbeDyOifBMGZ1ImmOGPfmMwsLD21+VHHMSSligcrA+ njFbUXQwDPx1KqDQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0036C136A5; Mon, 18 Mar 2024 17:07:25 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cYsDAM10+GUtdAAAD6G6ig (envelope-from ); Mon, 18 Mar 2024 17:07:24 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 997CCA07D9; Mon, 18 Mar 2024 18:07:24 +0100 (CET) Date: Mon, 18 Mar 2024 18:07:24 +0100 From: Jan Kara To: Kemeng Shi Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, tim.c.chen@linux.intel.com, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 1/6] fs/writeback: avoid to writeback non-expired inode in kupdate writeback Message-ID: <20240318170724.zatj2bgfv36fkkos@quack3> References: <20240228091958.288260-1-shikemeng@huaweicloud.com> <20240228091958.288260-2-shikemeng@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240228091958.288260-2-shikemeng@huaweicloud.com> X-Spam-Score: -3.80 X-Spamd-Result: default: False [-3.80 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_THREE(0.00)[3]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] X-Spam-Level: Authentication-Results: smtp-out2.suse.de; none X-Spam-Flag: NO On Wed 28-02-24 17:19:53, Kemeng Shi wrote: > In kupdate writeback, only expired inode (have been dirty for longer than > dirty_expire_interval) is supposed to be written back. However, kupdate > writeback will writeback non-expired inode left in b_io or b_more_io from > last wb_writeback. As a result, writeback will keep being triggered > unexpected when we keep dirtying pages even dirty memory is under > threshold and inode is not expired. To be more specific: > Assume dirty background threshold is > 1G and dirty_expire_centisecs is > > 60s. When we running fio -size=1G -invalidate=0 -ioengine=libaio > --time_based -runtime=60... (keep dirtying), the writeback will keep > being triggered as following: > wb_workfn > wb_do_writeback > wb_check_background_flush > /* > * Wb dirty background threshold starts at 0 if device was idle and > * grows up when bandwidth of wb is updated. So a background > * writeback is triggered. > */ > wb_over_bg_thresh > /* > * Dirtied inode will be written back and added to b_more_io list > * after slice used up (because we keep dirtying the inode). > */ > wb_writeback > > Writeback is triggered per dirty_writeback_centisecs as following: > wb_workfn > wb_do_writeback > wb_check_old_data_flush > /* > * Write back inode left in b_io and b_more_io from last wb_writeback > * even the inode is non-expired and it will be added to b_more_io > * again as slice will be used up (because we keep dirtying the > * inode) > */ > wb_writeback > > Fix this by moving non-expired inode to dirty list instead of more io > list for kupdate writeback in requeue_inode. > > Test as following: > /* make it more easier to observe the issue */ > echo 300000 > /proc/sys/vm/dirty_expire_centisecs > echo 100 > /proc/sys/vm/dirty_writeback_centisecs > /* create a idle device */ > mkfs.ext4 -F /dev/vdb > mount /dev/vdb /bdi1/ > /* run buffer write with fio */ > fio -name test -filename=/bdi1/file -size=800M -ioengine=libaio -bs=4K \ > -iodepth=1 -rw=write -direct=0 --time_based -runtime=60 -invalidate=0 > > Fio result before fix (run three tests): > 1360MB/s > 1329MB/s > 1455MB/s > > Fio result after fix (run three tests): > 1737MB/s > 1729MB/s > 1789MB/s > > Writeback for non-expired inode is gone as expeted. Observe this with trace > writeback_start and writeback_written as following: > echo 1 > /sys/kernel/debug/tracing/events/writeback/writeback_start/enab > echo 1 > /sys/kernel/debug/tracing/events/writeback/writeback_written/enable > cat /sys/kernel/tracing/trace_pipe > > Signed-off-by: Kemeng Shi Looks good. Feel free to add: Reviewed-by: Jan Kara Honza > --- > fs/fs-writeback.c | 13 ++++++++++--- > 1 file changed, 10 insertions(+), 3 deletions(-) > > diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c > index 5ab1aaf805f7..4e6166e07eaf 100644 > --- a/fs/fs-writeback.c > +++ b/fs/fs-writeback.c > @@ -1561,7 +1561,8 @@ static void inode_sleep_on_writeback(struct inode *inode) > * thread's back can have unexpected consequences. > */ > static void requeue_inode(struct inode *inode, struct bdi_writeback *wb, > - struct writeback_control *wbc) > + struct writeback_control *wbc, > + unsigned long dirtied_before) > { > if (inode->i_state & I_FREEING) > return; > @@ -1594,7 +1595,8 @@ static void requeue_inode(struct inode *inode, struct bdi_writeback *wb, > * We didn't write back all the pages. nfs_writepages() > * sometimes bales out without doing anything. > */ > - if (wbc->nr_to_write <= 0) { > + if (wbc->nr_to_write <= 0 && > + !inode_dirtied_after(inode, dirtied_before)) { > /* Slice used up. Queue for next turn. */ > requeue_io(inode, wb); > } else { > @@ -1862,6 +1864,11 @@ static long writeback_sb_inodes(struct super_block *sb, > unsigned long start_time = jiffies; > long write_chunk; > long total_wrote = 0; /* count both pages and inodes */ > + unsigned long dirtied_before = jiffies; > + > + if (work->for_kupdate) > + dirtied_before = jiffies - > + msecs_to_jiffies(dirty_expire_interval * 10); > > while (!list_empty(&wb->b_io)) { > struct inode *inode = wb_inode(wb->b_io.prev); > @@ -1967,7 +1974,7 @@ static long writeback_sb_inodes(struct super_block *sb, > spin_lock(&inode->i_lock); > if (!(inode->i_state & I_DIRTY_ALL)) > total_wrote++; > - requeue_inode(inode, tmp_wb, &wbc); > + requeue_inode(inode, tmp_wb, &wbc, dirtied_before); > inode_sync_complete(inode); > spin_unlock(&inode->i_lock); > > -- > 2.30.0 > -- Jan Kara SUSE Labs, CR