Received-SPF: pass (google.com: domain of linux-kernel+bounces-10314-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Fri, 22 Dec 2023 22:11:43 -0800 (PST)
From: David Rientjes <rientjes@google.com>
To: Chris Li <chrisl@kernel.org>
cc: Andrew Morton <akpm@linux-foundation.org>, linux-kernel@vger.kernel.org, 
    linux-mm@kvack.org, Wei Xu <weixugc@google.com>, 
    Yu Zhao <yuzhao@google.com>, Greg Thelen <gthelen@google.com>, 
    Chun-Tse Shao <ctshao@google.com>, Suren Baghdasaryan <surenb@google.com>, 
    Yosry Ahmed <yosryahmed@google.com>, Brain Geffon <bgeffon@google.com>, 
    Minchan Kim <minchan@kernel.org>, Michal Hocko <mhocko@suse.com>, 
    Mel Gorman <mgorman@techsingularity.net>, 
    Huang Ying <ying.huang@intel.com>, Nhat Pham <nphamcs@gmail.com>, 
    Johannes Weiner <hannes@cmpxchg.org>, Kairui Song <kasong@tencent.com>, 
    Zhongkun He <hezhongkun.hzk@bytedance.com>, 
    Kemeng Shi <shikemeng@huaweicloud.com>, Barry Song <v-songbaohua@oppo.com>, 
    Hugh Dickins <hughd@google.com>
Subject: Re: [PATCH] mm: swap: async free swap slot cache entries
In-Reply-To: <ZYYY1VBKdLHH-Kl3@google.com>
Message-ID: <f6bf2c8d-c37a-dab7-8ef8-38a35240edb6@google.com>
References: <20231221-async-free-v1-1-94b277992cb0@kernel.org> <20231222115208.ab4d2aeacdafa4158b14e532@linux-foundation.org> <ZYYY1VBKdLHH-Kl3@google.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 22 Dec 2023, Chris Li wrote:

> On Fri, Dec 22, 2023 at 11:52:08AM -0800, Andrew Morton wrote:
> > On Thu, 21 Dec 2023 22:25:39 -0800 Chris Li <chrisl@kernel.org> wrote:
> > 
> > > We discovered that 1% swap page fault is 100us+ while 50% of
> > > the swap fault is under 20us.
> > > 
> > > Further investigation show that a large portion of the time
> > > spent in the free_swap_slots() function for the long tail case.
> > > 
> > > The percpu cache of swap slots is freed in a batch of 64 entries
> > > inside free_swap_slots(). These cache entries are accumulated
> > > from previous page faults, which may not be related to the current
> > > process.
> > > 
> > > Doing the batch free in the page fault handler causes longer
> > > tail latencies and penalizes the current process.
> > > 
> > > Move free_swap_slots() outside of the swapin page fault handler into an
> > > async work queue to avoid such long tail latencies.
> > 
> > This will require a larger amount of total work than the current
> 
> Yes, there will be a tiny little bit of extra overhead to schedule the job
> on to the other work queue.
> 

How do you quantify the impact of the delayed swap_entry_free()?

Since the free and memcg uncharge are now delayed, is there not the 
possibility that we stay under memory pressure for longer?  (Assuming at 
least some users are swapping because of memory pressure.)

I would assume that since the free and uncharge itself is delayed that in 
the pathological case we'd actually be swapping *more* until the async 
worker can run.