Received-SPF: pass (google.com: domain of linux-kernel+bounces-10034-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Fri, 22 Dec 2023 11:52:08 -0800
From: Andrew Morton <akpm@linux-foundation.org>
To: Chris Li <chrisl@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Wei =?ISO-8859-1?Q?X?=
 =?UTF-8?Q?u=EF=BF=BC?= <weixugc@google.com>, Yu =?UTF-8?Q?Zhao=EF=BF=BC?=
 <yuzhao@google.com>, Greg Thelen <gthelen@google.com>, Chun-Tse Shao
 <ctshao@google.com>, Suren =?UTF-8?Q?Baghdasaryan=EF=BF=BC?=
 <surenb@google.com>, Yosry =?UTF-8?Q?Ahmed=EF=BF=BC?=
 <yosryahmed@google.com>, Brain Geffon <bgeffon@google.com>, Minchan Kim
 <minchan@kernel.org>, Michal Hocko <mhocko@suse.com>, Mel Gorman
 <mgorman@techsingularity.net>, Huang Ying <ying.huang@intel.com>, Nhat Pham
 <nphamcs@gmail.com>, Johannes Weiner <hannes@cmpxchg.org>, Kairui Song
 <kasong@tencent.com>, Zhongkun He <hezhongkun.hzk@bytedance.com>, Kemeng
 Shi <shikemeng@huaweicloud.com>, Barry Song <v-songbaohua@oppo.com>
Subject: Re: [PATCH] mm: swap: async free swap slot cache entries
Message-Id: <20231222115208.ab4d2aeacdafa4158b14e532@linux-foundation.org>
In-Reply-To: <20231221-async-free-v1-1-94b277992cb0@kernel.org>
References: <20231221-async-free-v1-1-94b277992cb0@kernel.org>
Precedence: bulk
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Thu, 21 Dec 2023 22:25:39 -0800 Chris Li <chrisl@kernel.org> wrote:

> We discovered that 1% swap page fault is 100us+ while 50% of
> the swap fault is under 20us.
> 
> Further investigation show that a large portion of the time
> spent in the free_swap_slots() function for the long tail case.
> 
> The percpu cache of swap slots is freed in a batch of 64 entries
> inside free_swap_slots(). These cache entries are accumulated
> from previous page faults, which may not be related to the current
> process.
> 
> Doing the batch free in the page fault handler causes longer
> tail latencies and penalizes the current process.
> 
> Move free_swap_slots() outside of the swapin page fault handler into an
> async work queue to avoid such long tail latencies.

This will require a larger amount of total work than the current
scheme.  So we're trading that off against better latency.

Why is this a good tradeoff?