Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 1 Jun 2021 11:00:15 -0400
From:   Johannes Weiner <hannes@cmpxchg.org>
To:     Matthew Wilcox <willy@infradead.org>
Cc:     Huang Ying <ying.huang@intel.com>,
        Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Peter Xu <peterx@redhat.com>, Hugh Dickins <hughd@google.com>,
        Mel Gorman <mgorman@suse.de>, Rik van Riel <riel@surriel.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Michal Hocko <mhocko@kernel.org>,
        Dave Hansen <dave.hansen@intel.com>,
        Tim Chen <tim.c.chen@intel.com>
Subject: Re: [PATCH] mm: free idle swap cache page after COW
Message-ID: <YLZLf7MI11rzGI1B@cmpxchg.org>
References: <20210601053143.1380078-1-ying.huang@intel.com>
 <YLYef3i2OGseGbsS@casper.infradead.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YLYef3i2OGseGbsS@casper.infradead.org>
Precedence: bulk

On Tue, Jun 01, 2021 at 12:48:15PM +0100, Matthew Wilcox wrote:
> On Tue, Jun 01, 2021 at 01:31:43PM +0800, Huang Ying wrote:
> > With commit 09854ba94c6a ("mm: do_wp_page() simplification"), after
> > COW, the idle swap cache page (neither the page nor the corresponding
> > swap entry is mapped by any process) will be left in the LRU list,
> > even if it's in the active list or the head of the inactive list.  So,
> > the page reclaimer may take quite some overhead to reclaim these
> > actually unused pages.
> > 
> > To help the page reclaiming, in this patch, after COW, the idle swap
> > cache page will be tried to be freed.  To avoid to introduce much
> > overhead to the hot COW code path,
> > 
> > a) there's almost zero overhead for non-swap case via checking
> >    PageSwapCache() firstly.
> > 
> > b) the page lock is acquired via trylock only.
> > 
> > To test the patch, we used pmbench memory accessing benchmark with
> > working-set larger than available memory on a 2-socket Intel server
> > with a NVMe SSD as swap device.  Test results shows that the pmbench
> > score increases up to 23.8% with the decreased size of swap cache and
> > swapin throughput.
> 
> So 2 percentage points better than my original idea?  Sweet.
> 
> > diff --git a/mm/memory.c b/mm/memory.c
> > index 2b7ffcbca175..d44425820240 100644
> > --- a/mm/memory.c
> > +++ b/mm/memory.c
> > @@ -3104,6 +3104,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
> >  				munlock_vma_page(old_page);
> >  			unlock_page(old_page);
> >  		}
> > +		if (page_copied)
> > +			free_swap_cache(old_page);
> >  		put_page(old_page);
> >  	}
> >  	return page_copied ? VM_FAULT_WRITE : 0;
> 
> Why not ...
> 
> 		if (page_copied)
> 			free_page_and_swap_cache(old_page);
> 		else
> 			put_page(old_page);
> 
> then you don't need to expose free_swap_cache().  Or does the test for
> huge_zero_page mess this up?

It's free_page[s]_and_swap_cache() we should reconsider, IMO.

free_swap_cache() makes for a clean API function that does one thing,
and does it right. free_page_and_swap_cache() combines two independent
operations, which has the habit of accumulating special case-handling
for some callers that is unncessary overhead for others (Abstraction
Inversion anti-pattern).

For example, free_page_and_swap_cache() adds an is_huge_zero_page()
check around the put_page() for the tlb batching code. This isn't
needed here. AFAICS it is also unnecessary for the other callsite,
__collapse_huge_page_copy(), where context rules out zero pages.

The common put_page() in Huang's version also makes it slighly easier
to follow the lifetime of old_page.

So I'd say exposing free_swap_cache() is a good move, for this patch
and in general.