Date:   Tue, 31 Jan 2023 20:06:28 -0800
From:   Chris Goldsworthy <quic_cgoldswo@quicinc.com>
To:     Roman Gushchin <roman.gushchin@linux.dev>
CC:     Sukadev Bhattiprolu <quic_sukadev@quicinc.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Rik van Riel <riel@surriel.com>,
        Roman Gushchin <guro@fb.com>, Vlastimil Babka <vbabka@suse.cz>,
        Joonsoo Kim <js1304@gmail.com>,
        Minchan Kim <minchan@kernel.org>,
        Chris Goldsworthy <quic_cgoldswo@quicinc.com>,
        Georgi Djakov <quic_c_gdjako@quicinc.com>,
        <linux-mm@kvack.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] mm,page_alloc,cma: configurable CMA utilization
Message-ID: <20230201040628.GA3767@hu-cgoldswo-sd.qualcomm.com>
References: <20230131071052.GB19285@hu-sbhattip-lv.qualcomm.com>
 <Y9lZoI89Nw4bjjOZ@P9FQF9L96D.corp.robot.car>
 <20230131201001.GA8585@hu-sbhattip-lv.qualcomm.com>
 <Y9mraBHucYdnHXiS@P9FQF9L96D.corp.robot.car>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <Y9mraBHucYdnHXiS@P9FQF9L96D.corp.robot.car>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk

On Tue, Jan 31, 2023 at 03:59:36PM -0800, Roman Gushchin wrote:
> On Tue, Jan 31, 2023 at 12:10:01PM -0800, Sukadev Bhattiprolu wrote:
> > On Tue, Jan 31, 2023 at 10:10:40AM -0800, Roman Gushchin wrote:
> > > Hi Sukadev!
> > > 
> > > Can you, please, share a bit more details about your setup? E.g. what is
> > > the zone size, the cma area size and the value you want to set your sysctl to?
> > 
> > Hi Roman,
> > 
> > I currently have a device with 8GB Zone normal and 600MB of CMA. We have a
> > slightly different implementation and use up all the available CMA region.
> > i.e. going forward, we intend to set the ratio to 100 or even higher.


Hi Roman,

> It means you want allocations be always served from a cma region first?

Exactly.

> What's the point of it?

We're operating in a resource constrained environment, and we want to maximize
the amount of memory free / headroom for GFP_KERNEL allocations on our SoCs,
which are especially important for DMA allocations that use an IOMMU. We need a
large amount of CMA on our SoCs for various reasons (e.g. for devices not
upstream of an IOMMU), but whilst that CMA memory is not in use, we want to
route all GFP_MOVABLE allocations to the CMA regions, which will free up memory
for GFP_KERNEL allocations. 

> The idea behind the current formula is to keep cma regions free if there is
> a plenty of other free memory, otherwise treat it on par with other memory.

With the current approach, if we have a large amount of movable memory allocated
that has not gone into the CMA regions yet, and a DMA use case starts that
causes the above condition to be met, we would head towards OOM conditions when
we otherwise could have delayed this with this change.  Note that since we're
working on Android, there is a daemon built on top of PSI called LMKD that will
start killing things under memory pressure (before an OOM is actually reached)
in order to free up memory. This patch should then reduce kills accordingly for
a better user experience by keeping a larger set of background apps alive. When
a CMA allocation does occur and pages get migrated out, there is a similar
reduction in headroom (you probably already know this and know of the FB
equivalent made by Johannes Weiner). 

Thanks,

Chris.