Received-SPF: pass (google.com: domain of linux-kernel+bounces-110539-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Thu, 21 Mar 2024 19:07:04 +0100
From: Jan Kara <jack@suse.cz>
To: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Jan Kara <jack@suse.cz>, akpm@linux-foundation.org, tj@kernel.org,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org, willy@infradead.org,
	bfoster@redhat.com, dsterba@suse.com, mjguzik@gmail.com,
	dhowells@redhat.com, peterz@infradead.org
Subject: Re: [PATCH 0/6] Improve visibility of writeback
Message-ID: <20240321180704.lqmtdmd5tzsbuyyu@quack3>
References: <20240320110222.6564-1-shikemeng@huaweicloud.com>
 <20240320172240.7buswiv7zj2m5odg@quack3>
 <44e3b910-8b52-5583-f8a9-37105bf5e5b6@huaweicloud.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <44e3b910-8b52-5583-f8a9-37105bf5e5b6@huaweicloud.com>

On Thu 21-03-24 16:12:52, Kemeng Shi wrote:
> on 3/21/2024 1:22 AM, Jan Kara wrote:
> > On Wed 20-03-24 19:02:16, Kemeng Shi wrote:
> >> This series tries to improve visilibity of writeback. Patch 1 make
> >> /sys/kernel/debug/bdi/xxx/stats show writeback info of whole bdi
> >> instead of only writeback info in root cgroup. Patch 2 add a new
> >> debug file /sys/kernel/debug/bdi/xxx/wb_stats to show per wb writeback
> >> info. Patch 4 add wb_monitor.py to monitor basic writeback info
> >> of running system, more info could be added on demand. Rest patches
> >> are some random cleanups. More details can be found in respective
> >> patches. Thanks!
> >>
> >> Following domain hierarchy is tested:
> >>                 global domain (320G)
> >>                 /                 \
> >>         cgroup domain1(10G)     cgroup domain2(10G)
> >>                 |                 |
> >> bdi            wb1               wb2
> >>
> >> /* all writeback info of bdi is successfully collected */
> >> # cat /sys/kernel/debug/bdi/252:16/stats:
> >> BdiWriteback:              448 kB
> >> BdiReclaimable:        1303904 kB
> >> BdiDirtyThresh:      189914124 kB
> >> DirtyThresh:         195337564 kB
> >> BackgroundThresh:     32516508 kB
> >> BdiDirtied:            3591392 kB
> >> BdiWritten:            2287488 kB
> >> BdiWriteBandwidth:      322248 kBps
> >> b_dirty:                     0
> >> b_io:                        0
> >> b_more_io:                   2
> >> b_dirty_time:                0
> >> bdi_list:                    1
> >> state:                       1
> >>
> >> /* per wb writeback info is collected */
> >> # cat /sys/kernel/debug/bdi/252:16/wb_stats:
> >> cat wb_stats
> >> WbCgIno:                    1
> >> WbWriteback:                0 kB
> >> WbReclaimable:              0 kB
> >> WbDirtyThresh:              0 kB
> >> WbDirtied:                  0 kB
> >> WbWritten:                  0 kB
> >> WbWriteBandwidth:      102400 kBps
> >> b_dirty:                    0
> >> b_io:                       0
> >> b_more_io:                  0
> >> b_dirty_time:               0
> >> state:                      1
> >> WbCgIno:                 4284
> >> WbWriteback:              448 kB
> >> WbReclaimable:         818944 kB
> >> WbDirtyThresh:        3096524 kB
> >> WbDirtied:            2266880 kB
> >> WbWritten:            1447936 kB
> >> WbWriteBandwidth:      214036 kBps
> >> b_dirty:                    0
> >> b_io:                       0
> >> b_more_io:                  1
> >> b_dirty_time:               0
> >> state:                      5
> >> WbCgIno:                 4325
> >> WbWriteback:              224 kB
> >> WbReclaimable:         819392 kB
> >> WbDirtyThresh:        2920088 kB
> >> WbDirtied:            2551808 kB
> >> WbWritten:            1732416 kB
> >> WbWriteBandwidth:      201832 kBps
> >> b_dirty:                    0
> >> b_io:                       0
> >> b_more_io:                  1
> >> b_dirty_time:               0
> >> state:                      5
> >>
> >> /* monitor writeback info */
> >> # ./wb_monitor.py 252:16 -c
> >>                   writeback  reclaimable   dirtied   written    avg_bw
> >> 252:16_1                  0            0         0         0    102400
> >> 252:16_4284             672       820064   9230368   8410304    685612
> >> 252:16_4325             896       819840  10491264   9671648    652348
> >> 252:16                 1568      1639904  19721632  18081952   1440360
> >>
> >>
> >>                   writeback  reclaimable   dirtied   written    avg_bw
> >> 252:16_1                  0            0         0         0    102400
> >> 252:16_4284             672       820064   9230368   8410304    685612
> >> 252:16_4325             896       819840  10491264   9671648    652348
> >> 252:16                 1568      1639904  19721632  18081952   1440360
> >> ...
> > 
> > So I'm wondering: Are you implementing this just because this looks
> > interesting or do you have a real need for the functionality? Why?
> Hi Jan, I added debug files to test change in [1] which changes the way how
> dirty background threshold of wb is calculated. Without debug files, we could
> only monitor writeback to imply that threshold is corrected.
> In current patchset, debug info has not included dirty background threshold yet,
> I will add it when discution of calculation of dirty background threshold in [1]
> is done.
> The wb_monitor.py is suggested by Tejun in [2] to improve visibility of writeback.
> The script is more convenient than trace to monitor writeback behavior of the running
> system.

Thanks for the pointer. OK, I agree this is useful so let's have a look
into the code :)

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR