Received-SPF: pass (google.com: domain of linux-kernel+bounces-11138-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Precedence: bulk
MIME-Version: 1.0
References: <20231222102255.56993-1-ryncsn@gmail.com> <20231222102255.56993-2-ryncsn@gmail.com>
 <CAOUHufYtSQnrVBC6OEXpT=LJW=L4J1Uw1yXpPrzaUhPWauo_3A@mail.gmail.com>
In-Reply-To: <CAOUHufYtSQnrVBC6OEXpT=LJW=L4J1Uw1yXpPrzaUhPWauo_3A@mail.gmail.com>
From: Kairui Song <ryncsn@gmail.com>
Date: Tue, 26 Dec 2023 02:05:53 +0800
Message-ID: <CAMgjq7Bi19ou-c7rDZH+RMRMcV7Z49-xJh5KmFCfGy8XqCyREA@mail.gmail.com>
Subject: Re: [PATCH 1/3] mm, lru_gen: batch update counters on againg
To: Yu Zhao <yuzhao@google.com>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, 
	linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Yu Zhao <yuzhao@google.com> =E4=BA=8E2023=E5=B9=B412=E6=9C=8825=E6=97=A5=E5=
=91=A8=E4=B8=80 15:29=E5=86=99=E9=81=93=EF=BC=9A
>
> On Fri, Dec 22, 2023 at 3:24=E2=80=AFAM Kairui Song <ryncsn@gmail.com> wr=
ote:
> >
> > From: Kairui Song <kasong@tencent.com>
> >
> > When lru_gen is aging, it will update mm counters page by page,
> > which causes a higher overhead if age happens frequently or there
> > are a lot of pages in one generation getting moved.
> > Optimize this by doing the counter update in batch.
> >
> > Although most __mod_*_state has its own caches the overhead
> > is still observable.
> >
> > Tested in a 4G memcg on a EPYC 7K62 with:
> >
> >   memcached -u nobody -m 16384 -s /tmp/memcached.socket \
> >     -a 0766 -t 16 -B binary &
> >
> >   memtier_benchmark -S /tmp/memcached.socket \
> >     -P memcache_binary -n allkeys \
> >     --key-minimum=3D1 --key-maximum=3D16000000 -d 1024 \
> >     --ratio=3D1:0 --key-pattern=3DP:P -c 2 -t 16 --pipeline 8 -x 6
> >
> > Average result of 18 test runs:
> >
> > Before: 44017.78 Ops/sec
> > After:  44687.08 Ops/sec (+1.5%)
> >
> > Signed-off-by: Kairui Song <kasong@tencent.com>
> > ---
> >  mm/vmscan.c | 64 +++++++++++++++++++++++++++++++++++++++++++++--------
> >  1 file changed, 55 insertions(+), 9 deletions(-)
>
> Usually most reclaim activity happens in kswapd, e.g., from the
> MongoDB benchmark (--duration=3D900):
> pgscan_kswapd 11294317
> pgscan_direct 128
> And kswapd always has current->reclaim_state->mm_walk. So the
> following should bring the vast majority of the improvement (assuming
> it's not noise) with far less code change:

Hi Yu,

This won't work for the fault path (eg. the memtier test):
Samples: 30K of event 'cycles', Event count (approx.): 69411674954
  Children      Self  Command          Shared Object               Symbol
-   85.95%     0.69%  memcached        [kernel.vmlinux]            [k]
asm_exc_page_fault
   - 85.25% asm_exc_page_fault
      - 85.00% exc_page_fault
         - 84.81% do_user_addr_fault
            - 84.01% handle_mm_fault
               - 83.70% __handle_mm_fault
                  - 82.57% do_swap_page
                     - 61.66% mem_cgroup_swapin_charge_folio
                        - 61.11% charge_memcg
                           - 60.76% try_charge_memcg
                              - 60.68% try_to_free_mem_cgroup_pages
                                   do_try_to_free_pages
                                 - shrink_node
                                    - 60.51% shrink_lruvec
                                       - 60.45% try_to_shrink_lruvec
                                          + 60.42% evict_folios
                     + 10.00% __swap_entry_free
                     + 3.81% swap_read_folio_bdev_sync
                     + 1.49% __pte_offset_map_lock
                     + 0.92% swap_cache_get_folio
                     + 0.80% folio_add_lru
                     + 0.75% vma_alloc_folio
                     + 0.60% swap_read_folio
                  + 0.73% do_anonymous_page
              0.54% lock_vma_under_rcu

And:
sudo cat /sys/kernel/debug/lru_gen_full | grep -A 25 benchmark
memcg    72 /benchmark
 node     0
        218       3283          1x          0x
                     0          0           0           0           0
         0           0
                     1          0           0           0           0
         0           0
                     2          0           0           0           0
         0           0
                     3          0           0           0           0
         0           0
                                0           0           0           0
         0           0
        219       2472       2756           0
                     0      14775r     303395e          0p          2r
         2e          0p
                     1          0r          0e          0p          0r
         0e          0p
                     2          0r          0e          0p          0r
         0e          0p
                     3          0r          0e      15262p          0r
         0e          0p
                                0           0           0           0
         0           0
        220       1652     456032          22
                     0          0           0           0           0
         0           0
                     1          0           0           0           0
         0           0
                     2          0           0           0           0
         0           0
                     3          0           0           0           0
         0           0
                                0           0           0           0
         0           0
        221        808     578570          13
                     0      15665R     309071T          0           0R
         1T          0
                     1          0R          0T          0           0R
         0T          0
                     2          0R          0T          0           0R
         0T          0
                     3          0R      15364T          0           0R
         0T          0
                          9191594L    3532525O    2425411Y      94393N
     18515F      10578A

It ages fast.

It's hard to share the code with mm_walk, because in next patch, it
tries to move the pages in bulk, there is no such logic for mm_walk.

It's not very effective with this benchmark indeed, I'll update with
some other tests.