Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20200818184122.29C415DF@viggo.jf.intel.com> <20200818184131.C972AFCC@viggo.jf.intel.com>
 <87lfi9wxk9.fsf@yhuang-dev.intel.com> <6a378a57-a453-0318-924b-05dfa0a10c1f@intel.com>
In-Reply-To: <6a378a57-a453-0318-924b-05dfa0a10c1f@intel.com>
From:   Yang Shi <shy828301@gmail.com>
Date:   Thu, 20 Aug 2020 09:26:57 -0700
Message-ID: <CAHbLzkrjxm38VV+ibQxoQkC4nW7F13aJcL5RypUchX30rqUstA@mail.gmail.com>
Subject: Re: [RFC][PATCH 5/9] mm/migrate: demote pages during reclaim
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     "Huang, Ying" <ying.huang@intel.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Yang Shi <yang.shi@linux.alibaba.com>,
        David Rientjes <rientjes@google.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Linux-MM <linux-mm@kvack.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, Aug 20, 2020 at 8:22 AM Dave Hansen <dave.hansen@intel.com> wrote:
>
> On 8/20/20 1:06 AM, Huang, Ying wrote:
> >> +    /* Migrate pages selected for demotion */
> >> +    nr_reclaimed += demote_page_list(&ret_pages, &demote_pages, pgdat, sc);
> >> +
> >>      pgactivate = stat->nr_activate[0] + stat->nr_activate[1];
> >>
> >>      mem_cgroup_uncharge_list(&free_pages);
> >> _
> > Generally, it's good to batch the page migration.  But one side effect
> > is that, if the pages are failed to be migrated, they will be placed
> > back to the LRU list instead of falling back to be reclaimed really.
> > This may cause some issue in some situation.  For example, if there's no
> > enough space in the PMEM (slow) node, so the page migration fails, OOM
> > may be triggered, because the direct reclaiming on the DRAM (fast) node
> > may make no progress, while it can reclaim some pages really before.
>
> Yes, agreed.

Kind of. But I think that should be transient and very rare. The
kswapd on pmem nodes will be waken up to drop pages when we try to
allocate migration target pages. It should be very rare that there is
not reclaimable page on pmem nodes.

>
> There are a couple of ways we could fix this.  Instead of splicing
> 'demote_pages' back into 'ret_pages', we could try to get them back on
> 'page_list' and goto the beginning on shrink_page_list().  This will
> probably yield the best behavior, but might be a bit ugly.
>
> We could also add a field to 'struct scan_control' and just stop trying
> to migrate after it has failed one or more times.  The trick will be
> picking a threshold that doesn't mess with either the normal reclaim
> rate or the migration rate.

In my patchset I implemented a fallback mechanism via adding a new
PGDAT_CONTENDED node flag. Please check this out:
https://patchwork.kernel.org/patch/10993839/.

Basically the PGDAT_CONTENDED flag will be set once migrate_pages()
return -ENOMEM which indicates the target pmem node is under memory
pressure, then it would fallback to regular reclaim path. The flag
would be cleared by clear_pgdat_congested() once the pmem node memory
pressure is gone.

We already use node flags to indicate the state of node in reclaim
code, i.e. PGDAT_WRITEBACK, PGDAT_DIRTY, etc. So, adding a new flag
sounds more straightforward to me IMHO.

>
> This is on my list to fix up next.
>