Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Pankaj Suryawanshi <pankaj.suryawanshi@einfochips.com>
To:     Kirill Tkhai <ktkhai@virtuozzo.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        Michal Hocko <mhocko@kernel.org>,
        "aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "minchan@kernel.org" <minchan@kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "khandual@linux.vnet.ibm.com" <khandual@linux.vnet.ibm.com>
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
Thread-Topic: [External] Re: vmscan: Reclaim unevictable pages
Thread-Index: AQHU3WaYCAMWaFadm0e/0+hd2XPYGaYRGYZ/gAAG5oCAAAD82oAAAx4AgAABDq0=
Date:   Mon, 18 Mar 2019 09:59:00 +0000
Message-ID: <SG2PR02MB309869FC3A436C71B50FA57BE8470@SG2PR02MB3098.apcprd02.prod.outlook.com>
References: <SG2PR02MB3098A05E09B0D3F3CB1C3B9BE84B0@SG2PR02MB3098.apcprd02.prod.outlook.com>
 <SG2PR02MB3098B751EC6B8E32806A42FBE84B0@SG2PR02MB3098.apcprd02.prod.outlook.com>
 <20190314084120.GF7473@dhcp22.suse.cz>
 <SG2PR02MB309894F6D7DF9148846088F3E84B0@SG2PR02MB3098.apcprd02.prod.outlook.com>
 <226a92b9-94c5-b859-c54b-3aacad3089cc@virtuozzo.com>
 <SG2PR02MB3098299456FB6AE2FD822C4CE84B0@SG2PR02MB3098.apcprd02.prod.outlook.com>
 <SG2PR02MB30988333AD658F8124070ABEE84B0@SG2PR02MB3098.apcprd02.prod.outlook.com>
 <SG2PR02MB3098AB587F4BFCD6B9D042FDE8440@SG2PR02MB3098.apcprd02.prod.outlook.com>
 <SG2PR02MB3098E6F2C4BAEB56AE071EDCE8440@SG2PR02MB3098.apcprd02.prod.outlook.com>
 <0b86dbca-cbc9-3b43-e3b9-8876bcc24f22@suse.cz>
 <SG2PR02MB309841EA4764E675D4649139E8470@SG2PR02MB3098.apcprd02.prod.outlook.com>
 <56862fc0-3e4b-8d1e-ae15-0df32bf5e4c0@virtuozzo.com>
 <SG2PR02MB3098EEAF291BFD72F4163936E8470@SG2PR02MB3098.apcprd02.prod.outlook.com>,<4c05dda3-9fdf-e357-75ed-6ee3f25c9e52@virtuozzo.com>
In-Reply-To: <4c05dda3-9fdf-e357-75ed-6ee3f25c9e52@virtuozzo.com>
Accept-Language: en-GB, en-US
Content-Language: en-GB
received-spf: None (protection.outlook.com: einfochips.com does not designate
 permitted sender hosts)
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: c2c74ccd-776a-4dfa-6bfa-08d6ab885856
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Mar 2019 09:59:00.2882
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 0adb040b-ca22-4ca6-9447-ab7b049a22ff
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2PR02MB3734
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

 =20
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 18 March 2019 15:17:56
To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.i=
bm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; k=
handual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
=A0=20

On 18.03.2019 12:43, Pankaj Suryawanshi wrote:
> Hi Kirill Tkhai,
>

Please, do not top posting:  https://kernelnewbies.org/mailinglistguideline=
s

Okay.

mailinglistguidelines - Linux Kernel Newbies
kernelnewbies.org
Set of FAQs for kernelnewbies mailing list. If you are new to this list ple=
ase read this page before you go on your quest for squeezing all the knowle=
dge from fellow members.

> Please see mm/vmscan.c in which it first added to list and than throw the=
 error :
> -------------------------------------------------------------------------=
-------------------------
> keep:
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 list_add(&page->lru, &ret=
_pages);
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 VM_BUG_ON_PAGE(PageLRU(pa=
ge) || PageUnevictable(page), page);
> -------------------------------------------------------------------------=
--------------------------
>=20
> Before throwing error, pages are added to list, this is under iteration o=
f shrink_page_list().

I say about about the list, which is passed to shrink_page_list() as first =
argument.
Did you mean candidate list which is passed to shrink_page_list().

shrink_inactive_list()
{
=A0=A0=A0=A0=A0=A0=A0 isolate_lru_pages(&page_list); // <-- you can't obtai=
n unevictable pages here.
=A0=A0=A0=A0=A0=A0=A0 shrink_page_list(&page_list);
}

below is the overview of flow of calls for your information.

cma_alloc() ->
alloc_contig_range() ->
start_isolate_page_range() ->
__alloc_contig_migrate_range() ->
isolate_migratepages_range() ->
reclaim_clean_pages_from_list() ->
shrink_page_list()
=A0
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> Sent: 18 March 2019 15:03:15
> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux=
.ibm.com
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org;=
 khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
> =A0=20
>=20
> Hi, Pankaj,
>=20
> On 18.03.2019 12:09, Pankaj Suryawanshi wrote:
>>
>> Hello
>>
>> shrink_page_list() returns , number of pages reclaimed, when pages is un=
evictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),=
page);
>=20
> the general idea is shrink_page_list() can't iterate PageUnevictable() pa=
ges.
> PageUnevictable() pages are never being added to lists, which shrink_page=
_list()
> uses for iteration. Also, a page can't be marked as PageUnevictable(), wh=
en
> it's attached to a shrinkable list.
>=20
> So, the problem should be somewhere outside shrink_page_list().
>=20
> I won't suggest you something about CMA, since I haven't dived in that co=
de.
>=20
>> We can add the unevictable pages in reclaim list in shrink_page_list(), =
return total number of reclaim pages including unevictable pages, let the c=
aller handle unevictable pages.
>>
>> I think the problem is shrink_page_list is awkard. If page is unevictabl=
e it goto activate_locked->keep_locked->keep lables, keep lable list_add th=
e unevictable pages and throw the VM_BUG instead of passing it to caller wh=
ile it relies on caller for non-reclaimed-non-unevictable=A0=A0  page's put=
back.
>> I think we can make it consistent so that shrink_page_list could return =
non-reclaimed pages via page_list and caller can handle it. As an advance, =
it could try to migrate mlocked pages without retrial.
>>
>>
>> Below is the issue of CMA_ALLOC of large size buffer : (Kernel version -=
 4.14.65 (On Android pie [ARM])).
>>
>> [=A0=A0 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || =
PageUnevictable(page))
>> [=A0=A0 24.726949] page->mem_cgroup:bd008c00
>> [=A0=A0 24.730693] ------------[ cut here ]------------
>> [=A0=A0 24.735304] kernel BUG at mm/vmscan.c:1350!
>> [=A0=A0 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>>
>>
>> Below is the patch which solved this issue :
>>
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..12ac353 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_he=
ad *page_list,
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sc->nr_scanned++;
>> =A0
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (unlikely(!page_evictab=
le(page)))
>> -=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 goto=
 activate_locked;
>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 goto cu=
ll_mlocked;
>> =A0
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (!sc->may_unmap && page=
_mapped(page))
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 go=
to keep_locked;
>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list=
_head *page_list,
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } else
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 li=
st_add(&page->lru, &free_pages);
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 continue;
>> -
>> +cull_mlocked:
>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (PageSwapCache(page))
>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 t=
ry_to_free_swap(page);
>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 unlock_page(page);
>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 list_add(&page->lru, &ret=
_pages);
>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 continue;
>> =A0activate_locked:
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 /* Not a candidate for swa=
pping, so reclaim swap space. */
>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (PageSwapCache(page) &&=
 (mem_cgroup_swap_full(page) ||
>>
>>
>>
>>
>> It fixes the below issue.
>>
>> 1. Large size buffer allocation using cma_alloc successful with unevicta=
ble pages.
>>
>> cma_alloc of current kernel will fail due to unevictable page
>>
>> Please let me know if anything i am missing.
>>
>> Regards,
>> Pankaj
>> =A0=A0=20
>> From: Vlastimil Babka <vbabka@suse.cz>
>> Sent: 18 March 2019 14:12:50
>> To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.i=
bm.com
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org=
; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>> =A0=20
>>
>> On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>>>
>>> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
>>
>> Can you send a proper patch with changelog explaining the change? I
>> don't know the context of this thread.
>>
>>> From: Pankaj Suryawanshi
>>> Sent: 15 March 2019 11:35:05
>>> To: Kirill Tkhai; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.or=
g
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>>
>>> [ cc linux-mm ]
>>>
>>>
>>> From: Pankaj Suryawanshi
>>> Sent: 14 March 2019 19:14:40
>>> To: Kirill Tkhai; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>>
>>> Hello ,
>>>
>>> Please ignore the curly braces, they are just for debugging.
>>>
>>> Below is the updated patch.
>>>
>>>
>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>> index be56e2e..12ac353 100644
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_h=
ead *page_list,
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sc->nr_scanned++;
>>>
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (unlikely(!page_evi=
ctable(page)))
>>> -=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 got=
o activate_locked;
>>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 goto c=
ull_mlocked;
>>>
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (!sc->may_unmap && =
page_mapped(page))
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0 goto keep_locked;
>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct lis=
t_head *page_list,
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } else
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0 list_add(&page->lru, &free_pages);
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 continue;
>>> -
>>> +cull_mlocked:
>>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (PageSwapCache(page))
>>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =
try_to_free_swap(page);
>>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 unlock_page(page);
>>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 list_add(&page->lru, &re=
t_pages);
>>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 continue;
>>> =A0 activate_locked:
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 /* Not a candidate for=
 swapping, so reclaim swap space. */
>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (PageSwapCache(page=
) && (mem_cgroup_swap_full(page) ||
>>>
>>>
>>>
>>> Regards,
>>> Pankaj
>>>
>>>
>>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>>> Sent: 14 March 2019 14:55:34
>>> To: Pankaj Suryawanshi; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>>>
>>>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>>>
>>>> No additional patches applied on top of vanilla.(Core MM).
>>>>
>>>> If=A0 I change in the vmscan.c as below patch, it will work.
>>>
>>> Sorry, but 4.14.65 does not have braces around trylock_page(),
>>> like in your patch below.
>>>
>>> See=A0=A0=A0=A0=A0=A0  https://git.kernel.org/pub/scm/linux/kernel/git/=
stable/linux.git/tree/mm/vmscan.c?h=3Dv4.14.65
>>>
>>> [...]
>>>
>>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>>> index be56e2e..2e51edc 100644
>>>>> --- a/mm/vmscan.c
>>>>> +++ b/mm/vmscan.c
>>>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct li=
st_head *page_list,
>>>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 page =3D lru_to_p=
age(page_list);
>>>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 list_del(&page->l=
ru);
>>>>>
>>>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (!trylock_page(pa=
ge)) {
>>>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0 goto keep;
>>>>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }
>>>
>>> ***********************************************************************=
***************************************************************************=
*********** eInfochips Business Disclaimer: This e-mail message and all att=
achments transmitted with it are=A0  intended=A0 solely for the use of the =
addressee and may contain legally privileged and confidential information. =
If the reader of this message is not the intended recipient, or an employee=
 or agent responsible for delivering this message to the intended recipient=
,=A0  you=A0 are hereby notified that any dissemination, distribution, copy=
ing, or other use of this message or its attachments is strictly prohibited=
. If you have received this message in error, please notify the sender imme=
diately by replying to this message and=A0  please=A0 delete it from your c=
omputer. Any views expressed in this message are those of the individual se=
nder unless otherwise stated. Company has taken enough precautions to preve=
nt the spread of viruses. However the company accepts no liability for any =
damage=A0  caused=A0 by any virus transmitted by this email. **************=
***************************************************************************=
********************************************************************
>>>
>>
>> =A0=A0=A0=A0=20
>>
>=A0=A0=A0=A0=20
>=20
    =