Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <48275d12-7c81-ba31-0ec6-c0623c143ef4@nutanix.com>
Date:   Mon, 7 Mar 2022 09:57:07 +0000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.6.1
Subject: Re: Regression of madvise(MADV_COLD) on shmem?
Content-Language: en-US
To:     Yu Zhao <yuzhao@google.com>, Minchan Kim <minchan@kernel.org>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Linux-MM <linux-mm@kvack.org>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        linux-api@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Tim Murray <timmurray@google.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        Suren Baghdasaryan <surenb@google.com>, dancol@google.com,
        Shakeel Butt <shakeelb@google.com>, sonnyrao@google.com,
        oleksandr@redhat.com, Hillf Danton <hdanton@sina.com>,
        lizeb@google.com, Dave Hansen <dave.hansen@intel.com>,
        "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>
References: <dd620dbd-6d71-7553-d1e9-95676ff12c82@nutanix.com>
 <YiKsUr1FQwmDP7V0@google.com>
 <CAOUHufZyrLrkEvFay+FddN5Ve3v_-JvNROHo9hEhsoVuCpwh0Q@mail.gmail.com>
 <CAOUHufaBi5Js6HXCO3R5nOo-+NiWajmDjM4mpx6H55JWS3auGQ@mail.gmail.com>
From:   Ivan Teterevkov <ivan.teterevkov@nutanix.com>
In-Reply-To: <CAOUHufaBi5Js6HXCO3R5nOo-+NiWajmDjM4mpx6H55JWS3auGQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?b0RUbFpTSHk4Uk0ra01xTkFuQUNEVGhYOFdNRGhhclBTLzRESzYzeTBUSktX?=
 =?utf-8?B?bkUxN3dMQXhid3Z3YU5Dbmg4YWNiaHVCOTAxZjZ2bTJZSFk0U1VTaml2V3VP?=
 =?utf-8?B?dDBNUGVwMDdHdnFhOEVEcmk3L1NvOVhzNm9OSXhRMlhsWlNrNyt3VWFqeHFh?=
 =?utf-8?B?RFBGZWlKY29XSWtQSlM4d1RLV1g3ZWtDVzZXbDZJSUZjMStvamc2cXNwTFZX?=
 =?utf-8?B?TTVJZlovZTRIdS9iOHZ5SHBlQno4aWZpOGpheFJ6ZEIrVWNDbXRtRjVpcnlp?=
 =?utf-8?B?dVVFam12aDdkaG1HSC9YektqSTBsczRmMm5KUVRTRmFRZFFxa1B6V0svTWdh?=
 =?utf-8?B?dHRaWEQvSGxlS2pQKzF3Y1VrZi9hUkQvYWpNQzZqMkMzcGo3ejBscEpZQmVs?=
 =?utf-8?B?VEdMS2FLSWFURmYvMjVpdnk1eCtWbStoWnRrbEkyTzc4TWRLQkdlTXJYK1ph?=
 =?utf-8?B?QkVBbEVMYnZHdDlhdVBSUENwZmwwQlp4dmJZS0RqdHBEOEhNdU8wS3N6TVZY?=
 =?utf-8?B?WjA1U1NYbXVPcHhpcHJOVXZka2ZRTm1iVXkxMTljaFRrdUJGS3NyTVg4ZEp5?=
 =?utf-8?B?Q1pwWG51WS9GUG1yS0hiMjNSZXNoVURPRWJZNG82MjkwZVBycWMzbEY4ZVFZ?=
 =?utf-8?B?ZklVRFpmVTY4ckRvYXZuMmhId2ZLc0NZazE3eXpWTDFNL0lLYmI5RmZXRCs3?=
 =?utf-8?B?OXMzTkNrM3JEVmdVRks0LzF4MHhXcEtYb01TNllYTjc1S1R6V2loUmo4S1Y2?=
 =?utf-8?B?anl3VzhTQ1V4Y3lNb2JnRTc5MW8xQ2JDNG44dUNKQlpyOHNxdGY5S0dUZ1pQ?=
 =?utf-8?B?Ykh5UUhZaUI3WGk2d3dGZDFXZy9aUUFjWk52aUVPZVBkSjZLSDNWc0pJRVl3?=
 =?utf-8?B?bHBGSll0R0ZRcGp0VVJuT1BMWUJwSUNhUEw4Z0ZZU1Q2UkdWMHM1eS8xYlJU?=
 =?utf-8?B?ZVNubHlFeFZmam5ZOExBOW9IV2N6cmNMT0gvSGNUSzcyM09FNkdDcGxLZzF2?=
 =?utf-8?B?eEVPaXZCaDB4QUQrUTVHZXJPUDJjREpyWS8rbGlEaUJVdWdoT243UzNOekdV?=
 =?utf-8?B?Sjk0OHlPZTZYTHV6c2JsOUlCekQ0Wk4vN1pVaGRkWmp4SVplL05EY29TWi9o?=
 =?utf-8?B?TTlZS1hNbHNBeklHdlRNZTYza3lPeVU2Z1czUkt1c0NIQUVUdTBkMzdvTVFS?=
 =?utf-8?B?c0RxazUwRHhBcEJMc0o2MWtPUko5VmU4MEY5cXlnTU9iZWQwdG1IWGs1TlhL?=
 =?utf-8?B?U2lzSktkQ2lWNHdUS2Q0czk2SCtYSEdNWlltRUYzZlg3UzRkU1FsUWdnNGZN?=
 =?utf-8?B?TVZtdmhXeE95ZUdNblJrZWk2cEJjdDNJNGhwdXdocEg3WTFseHh5bGRxSjFJ?=
 =?utf-8?B?b3Y2SG5xQVhSNkJJQTNqZCtnMlZ4dFNua1FkY2lGUzRBRnRwMkxWU0RYMzNF?=
 =?utf-8?B?bmxEaTJVbnVKcHFXaml5Tk1JeTFoeVhJd3lYQlFFZ3ZNMEU2YW1rTHZWUDJX?=
 =?utf-8?B?T1o3cjBPMkFTMmdubGhPVlE3WkFWQmxsQjh3QUZXd1hKYkwzRUtBblJQRXZ4?=
 =?utf-8?B?UlNHQ2ZzVFZsdXlXb211S2d4V1FWVmEvNDlvMGNic0VuUWpVYU5ERXhLeEVj?=
 =?utf-8?B?c3FDQVhqb09WV2ZnbHRvRDk4QnV5OXh1RWxXUmlwMWJCTE1ERWRldVlBSTlX?=
 =?utf-8?B?djJZanhOQ2tNQmRKV2hxcGFsMFBSK05iQUE3SzVtNDFOdWVuOEwwM0hsYkVJ?=
 =?utf-8?B?dFg2cVJkaWNCMll3dks3U0V6emFpMDlZbEFZVUdtYzVqLzNPUnVrL3ZMcHRR?=
 =?utf-8?B?WTFpZ3RVVVplMCtucERtSU9TMDNDRG1ibHhlVzRuZnVhcHI0cUcvbFZzZUE0?=
 =?utf-8?B?bW5HN0sxaFlJbnB5QmpWQXVSSmJUdlk2UCtkTGZleWJvNzdyMGJldllBamFk?=
 =?utf-8?B?R085TmlaY21mZjhPdFo1a1B5SjlhSHl3US9YdExvUnNqYVlHODJTaEM5MVFV?=
 =?utf-8?B?VWJHQm5CYmpieXdpNmN6bzlqSms4U1RYa24wOUx6cXh6ZTlielNaeFc0c1ZY?=
 =?utf-8?B?SEpWTjByN1M0TTNNN1NNU3pjZmVuWGJRZVA4TGV6WHBlanBnTWszb1drZDlT?=
 =?utf-8?B?WUZBZ2tERGI4eFEyaThmeVVYNVRsajllN2l4Q3cyZVl1ekdFM1lFbFlMb2Ju?=
 =?utf-8?B?cXFDTGQrVHIvZWhUUk1teHNOZGg1OXk5enFHL2l6bnFGSUZwTlBENjd5NkFW?=
 =?utf-8?B?cVk0cGthWEx2aUVHdUVKekxBd0NRPT0=?=
X-OriginatorOrg: nutanix.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 94686076-6d24-4632-b032-08da0020db8d
X-MS-Exchange-CrossTenant-AuthSource: CY4PR0201MB3460.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Mar 2022 09:57:15.1451
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: b8I8n+GISkpOJsPKW5EgCEK5kx74ltdmcRJ0EfcECf0fN5odE4BCNfkPXAYHLG71BqDvZ6c7ve9LLJ2awinWOZ+oGXZn6uR63OmtUcKNMyA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR02MB7819
X-Proofpoint-ORIG-GUID: 2oA293ugoFreLwohqQLoLUuiQUdp22BQ
X-Proofpoint-GUID: 2oA293ugoFreLwohqQLoLUuiQUdp22BQ
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.64.514
 definitions=2022-03-07_03,2022-03-04_01,2022-02-23_01
X-Proofpoint-Spam-Reason: safe
X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 05-Mar-22 9:49 AM, Yu Zhao wrote:
> On Sat, Mar 5, 2022 at 2:17 AM Yu Zhao <yuzhao@google.com> wrote:
>>
>> On Fri, Mar 4, 2022 at 5:18 PM Minchan Kim <minchan@kernel.org> wrote:
>>>
>>> On Fri, Mar 04, 2022 at 05:55:58PM +0000, Ivan Teterevkov wrote:
>>>> It makes me think there may be a regression in MADV_COLD. Please let me know
>>>> what do you reckon?
>>>
>>> Since deactive_page is called, I guess that's not a regression(?) from [1]
>>>
>>> Then, my random guess that you mentioned "Swap" as regression might be
>>> related to "workingset detection for anon page" since kernel changes balancing
>>> policy between file and anonymous LRU, which was merged into v5.8.
>>> It would be helpful to see if you try it on v5.7 and v5.8.
>>>
>>> [1] 12e967fd8e4e6, mm: do not allow MADV_PAGEOUT for CoW page
>>
>> Yes, I noticed this for a while. With commit b518154e59a ("mm/vmscan:
>> protect the workingset on anonymous LRU"), anon/shmem pages start on
>> the inactive lru, and in this case, deactive_page() is a NOP. Before
>> 5.9, anon/shmem pages start on the active lru, deactive_page() moves
>> zero pages in the test to the inactive lru and therefore protests the
>> "blob".
>>
>> This should fix the problem for your test case:
>>

Thank you, it does fix the test case in 5.17.0-rc6+.

>> I'll leave it to Minchan to decide whether this is worth fixing,
>> together with this one:
>>

I suppose add_page_to_lru_list_tail() makes madvise(MADV_COLD) more 
"insistent" with moving the pages at the end of the inactive LRU, but 
perhaps that's aligned with its purpose?

Cheers,
Ivan