Received-SPF: pass (google.com: domain of linux-kernel+bounces-47494-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Precedence: bulk
MIME-Version: 1.0
References: <20231221-async-free-v1-1-94b277992cb0@kernel.org>
 <CAJD7tkY6XF_rhnAzqhZ-mo8yw-W4hOjxFsbvH04oqVr0u8mOzQ@mail.gmail.com> <CAF8kJuOZO5C6J44U0CkU-Y9nGYnYiX4EQddjJGo+fQxh3BDVQg@mail.gmail.com>
In-Reply-To: <CAF8kJuOZO5C6J44U0CkU-Y9nGYnYiX4EQddjJGo+fQxh3BDVQg@mail.gmail.com>
From: Yosry Ahmed <yosryahmed@google.com>
Date: Wed, 31 Jan 2024 17:21:52 -0800
Message-ID: <CAJD7tkYEjr-o0hLVUFqNp+jCGeoYnzs+DsWL59O=FqrXDxW=UA@mail.gmail.com>
Subject: Re: [PATCH] mm: swap: async free swap slot cache entries
To: Chris Li <chrisl@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>, linux-kernel@vger.kernel.org, 
	linux-mm@kvack.org, =?UTF-8?B?V2VpIFh177+8?= <weixugc@google.com>, 
	=?UTF-8?B?WXUgWmhhb++/vA==?= <yuzhao@google.com>, 
	Greg Thelen <gthelen@google.com>, Chun-Tse Shao <ctshao@google.com>, 
	=?UTF-8?Q?Suren_Baghdasaryan=EF=BF=BC?= <surenb@google.com>, 
	Brain Geffon <bgeffon@google.com>, Minchan Kim <minchan@kernel.org>, Michal Hocko <mhocko@suse.com>, 
	Mel Gorman <mgorman@techsingularity.net>, Huang Ying <ying.huang@intel.com>, 
	Nhat Pham <nphamcs@gmail.com>, Johannes Weiner <hannes@cmpxchg.org>, Kairui Song <kasong@tencent.com>, 
	Zhongkun He <hezhongkun.hzk@bytedance.com>, Kemeng Shi <shikemeng@huaweicloud.com>, 
	Barry Song <v-songbaohua@oppo.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Jan 31, 2024 at 4:57=E2=80=AFPM Chris Li <chrisl@kernel.org> wrote:
>
> Hi Yosry,
>
> On Thu, Dec 28, 2023 at 7:34=E2=80=AFAM Yosry Ahmed <yosryahmed@google.co=
m> wrote:
> >
> > On Thu, Dec 21, 2023 at 10:25=E2=80=AFPM Chris Li <chrisl@kernel.org> w=
rote:
> > >
> > > We discovered that 1% swap page fault is 100us+ while 50% of
> > > the swap fault is under 20us.
> > >
> > > Further investigation show that a large portion of the time
> > > spent in the free_swap_slots() function for the long tail case.
> > >
> > > The percpu cache of swap slots is freed in a batch of 64 entries
> > > inside free_swap_slots(). These cache entries are accumulated
> > > from previous page faults, which may not be related to the current
> > > process.
> > >
> > > Doing the batch free in the page fault handler causes longer
> > > tail latencies and penalizes the current process.
> > >
> > > Move free_swap_slots() outside of the swapin page fault handler into =
an
> > > async work queue to avoid such long tail latencies.
> > >
> > > Testing:
> > >
> > > Chun-Tse did some benchmark in chromebook, showing that
> > > zram_wait_metrics improve about 15% with 80% and 95% confidence.
> > >
> > > I recently ran some experiments on about 1000 Google production
> > > machines. It shows swapin latency drops in the long tail
> > > 100us - 500us bucket dramatically.
> > >
> > > platform        (100-500us)             (0-100us)
> > > A               1.12% -> 0.36%          98.47% -> 99.22%
> > > B               0.65% -> 0.15%          98.96% -> 99.46%
> > > C               0.61% -> 0.23%          98.96% -> 99.38%
> >
> > I recall you mentioning that mem_cgroup_uncharge_swap() is the most
> > expensive part of the batched freeing. If that's the case, I am
> > curious what happens if we move that call outside of the batching
> > (i.e. once the swap entry is no longer used and will be returned to
> > the cache). This should amortize the cost of memcg uncharging and
> > reduce the tail fault latency without extra work. Arguably, it could
> > increase the average fault latency, but not necessarily in a
> > significant way.
> >
> > Ying pointed out something similar if I understand correctly (and
> > other operations that can be moved).
>
> If the goal is to let the swap fault return as soon as possible. Then
> the current approach is better.
> mem_cgroup_uncarge_swap() is only part of it. Not close to all of it.

I think there are a lot of operations that we can move out of
swapcache_free_entries():
- mem_cgroup_uncharge_swap()
- arch_swap_invalidate_page()
- zswap_invalidate()
- clear_shadow_from_swap_cache()
, and maybe others.

I am curious, if we move these operations from the batched freeing,
would this remove the increased tail latency and make it more
consistent, without doing extra work?

I believe this is what Ying was also asking about.

>
> >
> > Also, if we choose to follow this route, I think there we should flush
> > the async worker in drain_slots_cache_cpu(), right?
> Not sure I understand this part. The drain_slots_cache_cpu(), will
> free the entries already. The current lock around cache->free_lock
> should protect async workers accessing the entries. What do you mean
> by flushing?

Never mind. I just realized that the percpu caches are static, so they
are not freed in drain_slots_cache_cpu(). The NULL check in the async
worker should be enough protection.