Date:   Sun, 5 Feb 2023 21:22:28 -0800
From:   Chris Goldsworthy <quic_cgoldswo@quicinc.com>
To:     Minchan Kim <minchan@kernel.org>
CC:     Chris Goldsworthy <quic_cgoldswo@quicinc.com>,
        Roman Gushchin <roman.gushchin@linux.dev>,
        Sukadev Bhattiprolu <quic_sukadev@quicinc.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Rik van Riel <riel@surriel.com>,
        Roman Gushchin <guro@fb.com>, Vlastimil Babka <vbabka@suse.cz>,
        Joonsoo Kim <js1304@gmail.com>,
        Georgi Djakov <quic_c_gdjako@quicinc.com>,
        <linux-mm@kvack.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] mm,page_alloc,cma: configurable CMA utilization
Message-ID: <20230206052139.GA21897@hu-cgoldswo-sd.qualcomm.com>
References: <20230131071052.GB19285@hu-sbhattip-lv.qualcomm.com>
 <Y9lZoI89Nw4bjjOZ@P9FQF9L96D.corp.robot.car>
 <20230131201001.GA8585@hu-sbhattip-lv.qualcomm.com>
 <Y9mraBHucYdnHXiS@P9FQF9L96D.corp.robot.car>
 <20230201040628.GA3767@hu-cgoldswo-sd.qualcomm.com>
 <Y9r6LtMOPHfxr7UL@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <Y9r6LtMOPHfxr7UL@google.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk

On Wed, Feb 01, 2023 at 03:47:58PM -0800, Minchan Kim wrote:
> Hi Chris,
> 
> On Tue, Jan 31, 2023 at 08:06:28PM -0800, Chris Goldsworthy wrote:
> > We're operating in a resource constrained environment, and we want to maximize
> > the amount of memory free / headroom for GFP_KERNEL allocations on our SoCs,
> > which are especially important for DMA allocations that use an IOMMU. We need a
> > large amount of CMA on our SoCs for various reasons (e.g. for devices not
> > upstream of an IOMMU), but whilst that CMA memory is not in use, we want to
> > route all GFP_MOVABLE allocations to the CMA regions, which will free up memory
> > for GFP_KERNEL allocations. 
> 
> I like this patch for different reason but for the specific problem you
> mentioned, How about making reclaimer/compaction aware of the problem:
> 
> IOW, when the GFP_KERNEL/DMA allocation happens but not enough memory
> in the zones, let's migrates movable pages in those zones into CMA
> area/movable zone if they are plenty of free memory.
> 
> I guess you considered but did you observe some problems?

Hi Minchan,

This is not an approach we've considered. If you have a high-level idea of the
key parts of vmscan.c you'd need to touch to implement this, could you point me
to them?

I guess one drawback with this approach is that as soon as kswapd starts,
psi_memstall_enter() is called, which can eventually lead to LMKD running in
user space, which we want to minimize. One aim of what we're doing this is to
delay the calling of psi_memstall_enter().

It would be beneficial though on top of our change: if someone called
cma_alloc() and migrated out of the CMA regions, changing kswapd to behave like
this would move things back into the CMA regions after cma_release() is called
(instead of having to kill a user space process to have the CMA re-utilized upon
further user space actions).

Thanks,

Chris.