Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <1682679641-13652-1-git-send-email-zhaoyang.huang@unisoc.com>
 <ZE/y5wMggipQrKvb@P9FQF9L96D> <e0bd4476c7854cbfafea0ced9569220c@BJMBX01.spreadtrum.com>
 <ZFGIN5Gnc2W7LhmK@P9FQF9L96D.corp.robot.car> <CAGWkznGpi7YKYeNcKrOr=58=4VApq285ocCPVh22HJCWwCU85Q@mail.gmail.com>
 <ZFKMCBWvOqMk3g1V@P9FQF9L96D.corp.robot.car> <CAGWkznG5V+=onj9TsEhwh=Pxqt9+-KJAHLAoLJRMQ5-ysnwDgQ@mail.gmail.com>
In-Reply-To: <CAGWkznG5V+=onj9TsEhwh=Pxqt9+-KJAHLAoLJRMQ5-ysnwDgQ@mail.gmail.com>
From:   Zhaoyang Huang <huangzhaoyang@gmail.com>
Date:   Thu, 4 May 2023 14:30:22 +0800
Message-ID: <CAGWkznEQZUw8XwTuGvXXPKqwg-b+-9eZPDyT2bcWnjbV4fM-1g@mail.gmail.com>
Subject: =?UTF-8?Q?Re=3A_=E7=AD=94=E5=A4=8D=3A_=5BPATCH=5D_mm=3A_optimization_on_page_allocat?=
        =?UTF-8?Q?ion_when_CMA_enabled?=
To:     Roman Gushchin <roman.gushchin@linux.dev>
Cc:     =?UTF-8?B?6buE5pyd6ZizIChaaGFveWFuZyBIdWFuZyk=?= 
        <zhaoyang.huang@unisoc.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Roman Gushchin <guro@fb.com>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        =?UTF-8?B?546L56eRIChLZSBXYW5nKQ==?= <Ke.Wang@unisoc.com>,
        "fangzheng.zhang@unisoc.com" <fangzheng.zhang@unisoc.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Thu, May 4, 2023 at 2:23=E2=80=AFPM Zhaoyang Huang <huangzhaoyang@gmail.=
com> wrote:
>
> On Thu, May 4, 2023 at 12:30=E2=80=AFAM Roman Gushchin <roman.gushchin@li=
nux.dev> wrote:
> >
> > On Wed, May 03, 2023 at 03:58:21PM +0800, Zhaoyang Huang wrote:
> > > On Wed, May 3, 2023 at 6:01=E2=80=AFAM Roman Gushchin <roman.gushchin=
@linux.dev> wrote:
> > > >
> > > > On Tue, May 02, 2023 at 12:12:28PM +0000, =E9=BB=84=E6=9C=9D=E9=98=
=B3 (Zhaoyang Huang) wrote:
> > > > > > Hi Zhaoyang!
> > > > > >
> > > > > > On Fri, Apr 28, 2023 at 07:00:41PM +0800, zhaoyang.huang wrote:
> > > > > > > From: Zhaoyang Huang <zhaoyang.huang@unisoc.com>
> > > > > > >
> > > > > > > Please be notice bellowing typical scenario that commit 16867=
6649
> > > > > > > introduce, that is, 12MB free cma pages 'help' GFP_MOVABLE to=
 keep
> > > > > > > draining/fragmenting U&R page blocks until they shrink to 12M=
B without
> > > > > > > enter slowpath which against current reclaiming policy. This =
commit change
> > > > > > the criteria from hard coded '1/2'
> > > > > > > to watermark check which leave U&R free pages stay around WMA=
RK_LOW
> > > > > > > when being fallback.
> > > > > >
> > > > > > Can you, please, explain the problem you're solving in more det=
ails?
> > > > > I am trying to solve a OOM problem caused by slab allocation fail=
 as all free pages are MIGRATE_CMA by applying 168676649, which could help =
to reduce the fault ration from 12/20 to 2/20. I noticed it introduce the p=
henomenon which I describe above.
> > > > > >
> > > > > > If I understand your code correctly, you're effectively reducin=
g the use of cma
> > > > > > areas for movable allocations. Why it's good?
> > > > > Not exactly. In fact, this commit lead to the use of cma early th=
an it is now, which could help to protect U&R be 'stolen' by GFP_MOVABLE. I=
magine this scenario, 30MB total free pages composed of 10MB CMA and 20MB U=
&R, while zone's watermark low is 25MB. An GFP_MOVABLE allocation can keep =
stealing U&R pages(don't meet 1/2 criteria) without enter slowpath(zone_wat=
ermark_ok(WMARK_LOW) is true) until they shrink to 15MB. In my opinion, it =
makes more sense to have CMA take its duty to help movable allocation when =
U&R lower to certain zone's watermark instead of when their size become sma=
ller than CMA.
> > > > > > Also, this is a hot path, please, make sure you're not adding m=
uch overhead.
> > > > > I would like to take more thought.
> > > >
> > > > Got it, thank you for the explanation!
> > > >
> > > > How about the following approach (completely untested)?
> > > >
> > > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > > > index 6da423ec356f..4b50f497c09d 100644
> > > > --- a/mm/page_alloc.c
> > > > +++ b/mm/page_alloc.c
> > > > @@ -2279,12 +2279,13 @@ __rmqueue(struct zone *zone, unsigned int o=
rder, int migratetype,
> > > >         if (IS_ENABLED(CONFIG_CMA)) {
> > > >                 /*
> > > >                  * Balance movable allocations between regular and =
CMA areas by
> > > > -                * allocating from CMA when over half of the zone's=
 free memory
> > > > -                * is in the CMA area.
> > > > +                * allocating from CMA when over half of the zone's=
 easily
> > > > +                * available free memory is in the CMA area.
> > > >                  */
> > > >                 if (alloc_flags & ALLOC_CMA &&
> > > >                     zone_page_state(zone, NR_FREE_CMA_PAGES) >
> > > > -                   zone_page_state(zone, NR_FREE_PAGES) / 2) {
> > > > +                   (zone_page_state(zone, NR_FREE_PAGES) -
> > > > +                    zone->_watermark[WMARK_LOW]) / 2) {
> > > IMO, we should focus on non-cma area which trigger use of cma when
> > > they are lower than corresponding watermark(there is still
> > > WMARK_MIN/HIGH to deal with within slowpath)
> > > >                         page =3D __rmqueue_cma_fallback(zone, order=
);
> > > >                         if (page)
> > > >                                 return page;
> > > >
> > > > Basically the idea is to keep free space equally split between cma =
and non-cma areas.
> > > > Will it work for you?
> > > I don't think it makes sense to 'equally split' cma and non-cma areas
> > > over free space while cma could occupy various proportions in a singl=
e
> > > zone. This fixed 1/2 could lead to different situation on 20% or 50%
> > > cma occupation.
> >
> > Can you then, please, explain in details what you're proposing instead
> > and why it's better (not only in your case, but generally as well)?
> > For the context, my original usecase was cma size under 10G and
> > the total memory size between 32G and 128G.
>
> Let us look at the series of scenarios below with
> WMARK_LOW=3D25MB,WMARK_MIN=3D5MB(managed pages 1.9GB). We can know that
> 1. optimized 1/2 method start to use CMA since D which actually has
> caused U&R lower than WMARK_LOW (this could be deemed as against
> current memory policy, that is, U&R should either keeping stay  around
> WATERMARK_LOW or enter slowpath to do reclaiming)
> 2. it keep using CMA as long as free pages keep shrinking (this is
> against the original desire of balance cma & none-cma area)
>
> free_cma/free_pages(MB)        A(12/30)     B(10/25)     C(8/25)
> D(8/20)     E(8/16)     F(5/12)     F(5/10)   G(4/8)
> optimized 1/2                                N                 N
>          N             Y               Y                Y
> Y            Y
> !zone_watermark_ok                    Y                  Y
>    Y             N               N                N            Y
>      Y
sorry for the broken graph format of previous reply, repost it here.N
stands for not use CMA while Y stands for using

 free_cma/free_pages(MB)        A(12/30)     B(10/25)     C(8/25)
 optimized 1/2                                N                 N
          N
 !zone_watermark_ok                    Y                  Y                =
Y

D(8/20)     E(8/16)     F(5/12)     F(5/10)   G(4/8)
    Y               Y                Y            Y            Y
    N               N                N            Y           Y


> >
> > Looking at your original patch, I see that __if_use_cma_first() always =
returns
> > false if zone_watermark_ok() returns true. It worries me.
> ok. we could deal with the scenario here when free pages is higher
> than WMARK_HIGH
> >
> > Thanks!