Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 4 Jun 2020 09:36:57 -0700
From:   Matthew Wilcox <willy@infradead.org>
To:     Zi Yan <ziy@nvidia.com>
Cc:     Anshuman Khandual <anshuman.khandual@arm.com>, linux-mm@kvack.org,
        hughd@google.com, daniel.m.jordan@oracle.com,
        Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>,
        John Hubbard <jhubbard@nvidia.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH V2] mm/vmstat: Add events for THP migration without split
Message-ID: <20200604163657.GV19604@bombadil.infradead.org>
References: <1591243245-23052-1-git-send-email-anshuman.khandual@arm.com>
 <20200604113421.GU19604@bombadil.infradead.org>
 <CBF71911-6BB7-4AA7-AC0F-95AADBB45569@nvidia.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CBF71911-6BB7-4AA7-AC0F-95AADBB45569@nvidia.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, Jun 04, 2020 at 09:51:10AM -0400, Zi Yan wrote:
> On 4 Jun 2020, at 7:34, Matthew Wilcox wrote:
> > On Thu, Jun 04, 2020 at 09:30:45AM +0530, Anshuman Khandual wrote:
> >> +Quantifying Migration
> >> +=====================
> >> +Following events can be used to quantify page migration.
> >> +
> >> +- PGMIGRATE_SUCCESS
> >> +- PGMIGRATE_FAIL
> >> +- THP_MIGRATION_SUCCESS
> >> +- THP_MIGRATION_FAILURE
> >> +
> >> +THP_MIGRATION_FAILURE in particular represents an event when a THP could not be
> >> +migrated as a single entity following an allocation failure and ended up getting
> >> +split into constituent normal pages before being retried. This event, along with
> >> +PGMIGRATE_SUCCESS and PGMIGRATE_FAIL will help in quantifying and analyzing THP
> >> +migration events including both success and failure cases.
> >
> > First, I'd suggest running this paragraph through 'fmt'.  That way you
> > don't have to care about line lengths.
> >
> > Second, this paragraph doesn't really explain what I need to know to
> > understand the meaning of these numbers.  When Linux attempts to migrate
> > a THP, one of three things can happen:
> >
> >  - It is migrated as a single THP
> >  - It is migrated, but had to be split
> >  - Migration fails
> >
> > How do I turn these four numbers into an understanding of how often each
> > of those three situations happen?  And why do we need four numbers to
> > report three situations?
> >
> > Or is there something else that can happen?  If so, I'd like that explained
> > here too ;-)
> 
> PGMIGRATE_SUCCESS and PGMIGRATE_FAIL record a combination of different events,
> so it is not easy to interpret them. Let me try to explain them.

Thanks!  Very helpful explanation.

> 1. migrating only base pages: PGMIGRATE_SUCCESS and PGMIGRATE_FAIL just mean
> these base pages are migrated and fail to migrate respectively.
> THP_MIGRATION_SUCCESS and THP_MIGRATION_FAILURE should be 0 in this case.
> Simple.
> 
> 2. migrating only THPs:
> 	- PGMIGRATE_SUCCESS means THPs that are migrated and base pages
> 	(from the split of THPs) that are migrated,
> 
> 	- PGMIGRATE_FAIL means THPs that fail to migrate and base pages that fail to migrated.
> 
> 	- THP_MIGRATION_SUCCESS means THPs that are migrated.
> 
> 	- THP_MIGRATION_FAILURE means THPs that are split.
> 
> So PGMIGRATE_SUCCESS - THP_MIGRATION_SUCCESS means the number of migrated base pages,
> which are from the split of THPs.

Are you sure about that?  If I split a THP and each of those subpages
migrates, won't I then see PGMIGRATE_SUCCESS increase by 512?

> When it comes to analyze failed migration, PGMIGRATE_FAIL - THP_MIGRATION_FAILURE
> means the number of pages that are failed to migrate, but we cannot tell how many
> are base pages and how many are THPs.
> 
> 3. migrating base pages and THP:
> 
> The math should be very similar to the second case, except that
> a) from PGMIGRATE_SUCCESS - THP_MIGRATION_SUCCESS, we cannot tell how many are pages begin
> as base pages and how many are pages begin as THPs but become base pages after split;
> b) from PGMIGRATE_FAIL - THP_MIGRATION_FAILURE, an additional case,
> base pages that begin as base pages fail to migrate, is mixed into the number and we
> cannot tell three cases apart.

So why don't we just expose PGMIGRATE_SPLIT?  That would be defined as
the number of times we succeeded in migrating a THP but had to split it
to succeed.