Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Zi Yan <ziy@nvidia.com>
To:     Kefeng Wang <wangkefeng.wang@huawei.com>
Cc:     Naoya Horiguchi <naoya.horiguchi@nec.com>,
        Matthew Wilcox <willy@infradead.org>,
        Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, Huang Ying <ying.huang@intel.com>,
        David Hildenbrand <david@redhat.com>,
        Mike Kravetz <mike.kravetz@oracle.com>
Subject: Re: [PATCH 1/4] mm: migrate: use a folio in add_page_for_migration()
Date:   Mon, 07 Aug 2023 14:45:59 -0400
Message-ID: <5BBFF5D3-3416-4C0E-9FDD-655661657D67@nvidia.com>
In-Reply-To: <de0100e4-d673-428b-8d50-11ae2b7a9641@huawei.com>
References: <20230802095346.87449-1-wangkefeng.wang@huawei.com>
 <20230802095346.87449-2-wangkefeng.wang@huawei.com>
 <ZMpKYfNWA/jNgEuL@casper.infradead.org>
 <001ee9b0-ea25-a896-e3ae-9a9b05a46546@huawei.com>
 <ZMud3RreEpsvFKuA@casper.infradead.org>
 <fb2a22cf-14ae-3594-f5f3-8680c2100d70@huawei.com>
 <F2621E68-F36E-493C-8619-ADFE05050823@nvidia.com>
 <d184ba78-97d1-a264-fc31-87dfdbe6fdff@huawei.com>
 <de0100e4-d673-428b-8d50-11ae2b7a9641@huawei.com>
Content-Type: multipart/signed;
 boundary="=_MailMate_D1A869DF-D4D0-4CA6-8237-85B3F5D9848D_=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UHFsNmlnaUpoV0dWdHNaUzVrSUhSdmVYWUVzb1k4Um9DMlU1cC9CS09ZUGJI?=
 =?utf-8?B?TWRjRTJYVW9SNGxvbUYxOEhpZHk5L2g5c2htKzFuYjNiQ2JuVDhDaXdILzhu?=
 =?utf-8?B?b3JUZWUzRzdZN3cwRGR5MWJadXhPK0JWamI1VUU2NmNxaUdQUmpidklMOG83?=
 =?utf-8?B?dFVpWUxpUUl4WFNkQUNwcVlCR0U4VWxEeTdXR3NNUHlIb21wZGlkQzYzRERl?=
 =?utf-8?B?ZlgrNWRTcTlmYlc2ai9pbTV0NmhDSW5LNElEYzc2V1VlZEVSeFd2N1pXaUxq?=
 =?utf-8?B?amEzWllTVG0wWStiMERqM25kL3ZjcHd6Q0x4TFNQeHREL1hlNkhkSnBCeFdx?=
 =?utf-8?B?T3QwN0hrOTNWOG9PYXJYOWlPK1hUaU9SenpseHNLY0VBZnJmbjk4cDJDYldJ?=
 =?utf-8?B?MDgyajdiTGkwSklYTTlkT2ptZEZ2Qm0zWE9jT2FnWjFrLzdmQ0pBU1NDT1BX?=
 =?utf-8?B?UHNjeTRON0hFUVdSRlg2NmZCRWVhQmxtdWFjOVpuK0VLempBUFh4VlF1Wldm?=
 =?utf-8?B?UFlFbEU0UUZSUm9KdVVWbHh0L0w5Tm1pbHovVHFPVnExRSs3NkJiRCtVZmhR?=
 =?utf-8?B?OEZ5QldGQnRnbmpPUnZMSUFyLzZRMGIrc0FKb2hhSzdZblVTdXpRSzlHYjYx?=
 =?utf-8?B?YkVmOUY1cHBuUmQ0M1JNbzVhYURmQVJ4aGdUUk1CYm5vb0NBQllBcGY3ZmNa?=
 =?utf-8?B?RFBZSXJDWHVWeWZrc2hVbVF3bUx2aDJGTG9pTUZHc0hYWHdFU1k0bDlsUFhz?=
 =?utf-8?B?alhKRUpGNWI4RG1LWFVQQnRjOUNhUURjVG8wZk1Fbm4zRXQxM1JpZjFDWjBD?=
 =?utf-8?B?NkRsQVFtL1pMRCt4R1JhRFN5NHBrb1NkT2Mza2tWbmt5eUlhbnJyS2NEVVlq?=
 =?utf-8?B?RERiNGFBVVBFZkNnZG83R3hpWjJyMzdIUHg5WFJZZm1VUU11a0R3ZEVJbEZ0?=
 =?utf-8?B?cHZKMmo5c1N6VHZBNnVQa1dTU1hLZTZtc1o2QnI1NUZDRk9GL2tHMTFBTGtz?=
 =?utf-8?B?Y3RSblBWQXZFYUEzekVNdjVkdE9TUERxVjN5cUlNVW5JU21LNW1Rd3U3c1A4?=
 =?utf-8?B?MGdRY1E3S0YvWDJOTVozVXRoVCtkcGNnTjRxOXJpdVl6NHlycXZMNnAzcGxM?=
 =?utf-8?B?d2cvMFM4bHc2ZVhiMXFRaEZRbVgwK1N4MFdsWXR0OGVvYzdmbXBOU2tsbllw?=
 =?utf-8?B?Y2QxcnFuTmxKVmlqMnJ6andlT1UxaHQzUVJGVVBYSXIwNmp2RnpUQkRJeEpX?=
 =?utf-8?B?Um0vaGtJbmp1OVg0SnI3dmthSEQwYjBneHBMQ0QyMXJCRFRHZ1pVNExreDJV?=
 =?utf-8?B?QnBkVERrRjFXcVMwYVN1Z2VHL01JTXRTZGUwb2tmWktYTDZjKzJLeDV6aGo0?=
 =?utf-8?B?Q0EvMWxlMGMwajJMQnFhR2piM2VHbUdTWXFSV0dYQlN0YXRZS0RWRTRaZDFl?=
 =?utf-8?B?WTZLS1Bsbkx5RmdJbnR2Y1JhdHZVa0hSc1BtZE1vWEpQcUNRVDlDVlBDY0ph?=
 =?utf-8?B?bTQ4U1BkclBGS3YyaDk3TUZzVjQwNUlYR0lFYVc2SHl3SFdnT3NkZ0VOc2ln?=
 =?utf-8?B?SGl0YS9NRXkzYldoQ1drR0dJazQzcis2N244dldlMDZ4eXNncGtFa2RtSkhQ?=
 =?utf-8?B?U2wrbnFpSlJRZ05CWWtpa3VkQ3d3R25zSU01emhZLzhpZ2NKR2pXU1huYUlQ?=
 =?utf-8?B?cUlrZ0VYMGRlV0xBMitmdUFjODhJWmJvaFhKakQ3c1JBUHd0dVMxS1NRTis1?=
 =?utf-8?B?bldOWHh5SmtYS2xjS00vKzFPZXBxSzlPYm1rSGtydzlnWklPSXl1V3IvenJL?=
 =?utf-8?B?Mlc0ZDdEeVY4UzhzcXhzdlRjVEhVVFhVVVZ3cjkwbkZMRXp3MERlUzJoWUVz?=
 =?utf-8?B?Z3k4ekdKSk9QUFFxSC9tZm9KR3VXWDY1YVNkMVhYT0QzMXZwU3g4TmVaWGdK?=
 =?utf-8?B?bXRObGJlUkpwQzZ0T0xOZUxCVWJqYml3SzNPay8zYklPdXI1cFRYYkF2UGZx?=
 =?utf-8?B?N3FIUDVSNGpBcHlBeGtmMWo2OXdHVGJLanRlbmQxUDZYNnVGQnZBbzcyT0pw?=
 =?utf-8?B?a3d2TXdxU000QVV1aEtHZEhtZzBPODZoV0FJd1k2T1N5Mm53RDlZVkxXUys5?=
 =?utf-8?Q?0qPk=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2e5ec2b6-bc99-4e65-2731-08db97768be0
X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Aug 2023 18:46:01.2980
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +V/gj0Ead8+o3WyfzjuHRlA461Db5PCRE+Cn6NwEtx2m6n9DAif2Jxok2CjkUF1A
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6872
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

--=_MailMate_D1A869DF-D4D0-4CA6-8237-85B3F5D9848D_=
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 7 Aug 2023, at 8:20, Kefeng Wang wrote:

> Hi Zi Yan and Matthew and Naoya,
>
> On 2023/8/4 13:54, Kefeng Wang wrote:
>>
>>
>> On 2023/8/4 10:42, Zi Yan wrote:
>>> On 3 Aug 2023, at 21:45, Kefeng Wang wrote:
>>>
>>>> On 2023/8/3 20:30, Matthew Wilcox wrote:
>>>>> On Thu, Aug 03, 2023 at 03:13:21PM +0800, Kefeng Wang wrote:
>>>>>>
>
> ...
>
>>>>
>>>>
>>>> =C2=A0=C2=A0 if (PageHuge(page))=C2=A0 // page must be a hugetlb pag=
e
>>>> =C2=A0=C2=A0=C2=A0=C2=A0if (PageHead(page)) // page must be a head p=
age, not tail
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 isolate_hugetlb() // isolate the hugetlb page if head
>>>>
>>>> After using folio,
>>>>
>>>> =C2=A0=C2=A0 if (folio_test_hugetlb(folio)) // only check folio is h=
ugetlb or not
>>>>
>>>> I don't check the page is head or not, since the follow_page could
>>>> return a sub-page, so the check PageHead need be retained, right?
>>>
>>> Right. It will prevent the kernel from trying to isolate the same hug=
etlb page
>>> twice when two pages are in the same hugetlb folio. But looking at th=
e
>>> code, if you try to isolate an already-isolated hugetlb folio, isolat=
e_hugetlb()
>>> would return false, no error would show up. But it changes err value
>>> from -EACCES to -EBUSY and user will see a different page status than=
 before.
>>
>
> Before e66f17ff7177 ("mm/hugetlb: take page table lock in follow_huge_p=
md()")
> in v4.0, follow_page() will return NULL on tail page for Huagetlb page,=

> and move_pages() will return -ENOENT errno,but after that commit,
> -EACCES is returned, which not match the manual,
>
>>
>> When check man[1], the current -EACCES is not right, -EBUSY is not
>> precise but more suitable for this scenario,
>>
>>  =C2=A0=C2=A0=C2=A0=C2=A0 -EACCES
>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 The page is mapped by multiple processes and can be moved
>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 only if MPOL_MF_MOVE_ALL is specified.
>>
>>  =C2=A0=C2=A0=C2=A0=C2=A0 -EBUSY The page is currently busy and cannot=
 be moved.=C2=A0 Try again
>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 later.=C2=A0 This occurs if a page is undergoing I/O or another=

>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 kernel subsystem is holding a reference to the page.
>>  =C2=A0=C2=A0=C2=A0=C2=A0-ENOENT
>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 The page is not present.
>>
>>>
>>> I wonder why we do not have follow_folio() and returns -ENOENT error =
pointer
>>> when addr points to a non head page. It would make this patch more fo=
lio if
>>> follow_folio() can be used in place of follow_page(). One caveat is t=
hat
>>> user will see -ENOENT instead of -EACCES after this change.
>>>
>>
>> -ENOENT is ok, but maybe the man need to be updated too.
>
> According to above analysis, -ENOENT is suitable when introduce the
> follow_folio(), but when THP migrate support is introduced by
> e8db67eb0ded ("mm: migrate: move_pages() supports thp migration") in
> v4.14, the tail page will be turned into head page and return -EBUSY,
>
> So should we unify errno(maybe use -ENOENT) about the tail page?
>
>
>>
>>
>>
>> [1] https://man7.org/linux/man-pages/man2/move_pages.2.html

I think so. I think -EBUSY is more reasonable for tail pages. But there i=
s
some subtle difference between THP and hugetlb from current code:

For THP, compound_head() is used to get the head page for isolation, this=
 means
if user specifies a tail page address in move_pages(), the whole THP can =
be
migrated.

For hugetlb, only if user specifies the head page address of a hugetlb pa=
ge,
the hugetlb page will be migrated. Otherwise, an error would show up.

Cc Mike to help us clarify the expected behavior of hugetlb.

Hi Mike, what is the expected behavior, if a user tries to use move_pages=
()
to migrate a non head page of a hugetlb page?

--
Best Regards,
Yan, Zi

--=_MailMate_D1A869DF-D4D0-4CA6-8237-85B3F5D9848D_=
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename=signature.asc
Content-Type: application/pgp-signature; name=signature.asc

-----BEGIN PGP SIGNATURE-----

iQJDBAEBCgAtFiEE6rR4j8RuQ2XmaZol4n+egRQHKFQFAmTRO+cPHHppeUBudmlk
aWEuY29tAAoJEOJ/noEUByhUAE0P/149O0Wit/HL4Fo5mqNJlWR797qVstmIhuiu
wSKi8rZ3sBQKT5oaOIjmvCtKwO2ibvJsoOPXMY6OBvmXPaqfC2jNv//npuG7heZ0
IZ4qivf5+e8wl5BFPyYMjUanNaRvtIb86B9Qdq8UsmvlTEd4/H2BtJ9MCaLc4nH5
HC5tBEePcqYuRrGjWn9kAKHrsFJNUa52pqpNrPCHoa0P9mInONyp+v8MAZnpAlYd
fL6aOh9bmZusFTXhT23mlHtz3LXN9NrZvGWzaMne4riQ4FQwYsKpXK54XHSd+LpS
g1WoSOLS9tabfP5tZbh1Se5I98FgzZAPziWjgONYd5hb4jiULBR6FdDEwMuKtu9i
cfncM8OF0gwGtjxyOsVwX8lnmUwAXVnwGippPa4CMBfLmrE4+KadYkOqWynxme+U
Cl2DVtAL9xv6AWqbd1bG8itRUr0fF9BTE7s1Egzzd+WxWmD2Joc1o+FAypXKCFyn
io/5gNnIYLGquHt9kyPckGdD1qCzs3fjlEpFYAXc/prp392oz1Y9uTXTTZNG9ECm
i8qfg8CBaaDKty8zHQ+F0qOM+xqXwRqIfjh6Iqjv20vB2d2p0Ab7ZApFukKJpMD2
f3Lf/1Kitu7HtVCXIS12jsZ4TKPB4lXUWTqa+LQEF/l+Um4y9YbvUKLUav0d00ik
mGrku0bJ
=tggN
-----END PGP SIGNATURE-----

--=_MailMate_D1A869DF-D4D0-4CA6-8237-85B3F5D9848D_=--