Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <4ef41c10-61f4-558e-76b3-3ea517779985@oracle.com>
Date:   Mon, 9 May 2022 14:05:36 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.8.0
Subject: Re: [PATCH v2 2/3] mm: rmap: Fix CONT-PTE/PMD size hugetlb issue when
 migration
Content-Language: en-US
To:     Baolin Wang <baolin.wang@linux.alibaba.com>,
        akpm@linux-foundation.org, catalin.marinas@arm.com, will@kernel.org
Cc:     tsbogend@alpha.franken.de, James.Bottomley@HansenPartnership.com,
        deller@gmx.de, mpe@ellerman.id.au, benh@kernel.crashing.org,
        paulus@samba.org, hca@linux.ibm.com, gor@linux.ibm.com,
        agordeev@linux.ibm.com, borntraeger@linux.ibm.com,
        svens@linux.ibm.com, ysato@users.sourceforge.jp, dalias@libc.org,
        davem@davemloft.net, arnd@arndb.de,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        linux-ia64@vger.kernel.org, linux-mips@vger.kernel.org,
        linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
        linux-s390@vger.kernel.org, linux-sh@vger.kernel.org,
        sparclinux@vger.kernel.org, linux-arch@vger.kernel.org,
        linux-mm@kvack.org
References: <cover.1652002221.git.baolin.wang@linux.alibaba.com>
 <1ec8a987be1a5400e077260a300d0079564b1472.1652002221.git.baolin.wang@linux.alibaba.com>
From:   Mike Kravetz <mike.kravetz@oracle.com>
In-Reply-To: <1ec8a987be1a5400e077260a300d0079564b1472.1652002221.git.baolin.wang@linux.alibaba.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ajVISUdxUDdLZWNGN2NEYjRvQURuWUlqOHhNMEhCZi9NUTVGSmlwaEhESEpD?=
 =?utf-8?B?cHI2SHAwQ0JhMWVIUk02YWFueTZXdEQ3SGQyd2k2QVIxek5TeDV6VDduQngx?=
 =?utf-8?B?SnpaYjNxQ2tkb0FWVGkvZit0MkVHazlmNWZJY3hxUFVlZ2s2WWt5SG04a0x4?=
 =?utf-8?B?Q0FtZkc2dDdyZnJLQ0txNDU5cUhkVEZWVlZSUE9MRTRvY25LQ3BjUkxOREJF?=
 =?utf-8?B?RUt1SHNNRXM2bkNxdGFKMENJOFpoNEhubnpHR0ladVN0YXcyOXlHdnl1aHNS?=
 =?utf-8?B?R09palliZWlhVE5ReDBmajVnTXpNV0pTbjFDMEF3a3Z6UnZ4ZmkrNlZDKytY?=
 =?utf-8?B?cEx0cXNQR1R6cEdFOVpPMmtXaTZ4TjNWN2xiYkIxNHYwUFhpTXJGZFhTTXFD?=
 =?utf-8?B?YzhUcXdoaFlFRUo0Y3VMZy9jZ3NKVk9DZXd0a0lxTFM2OTBaZ0hRcW84MFJY?=
 =?utf-8?B?NUlBNkNRTHM4RFdmQk9tTXJBWXlvUktURVh5dC82MGxYUlZtdnN2WlFlYmEw?=
 =?utf-8?B?VEt3OWhjREFRaGZwejJPUFBoejlXM2l0VU55QXRLMUphTDhKekZIaVZGTnpQ?=
 =?utf-8?B?ODRFM0JXdXpZRzRlZThqVVZzU1dtRkNaNEVWZ0NoZVVlRVhlR01VSG1RLyt0?=
 =?utf-8?B?cW5MOFlwUjdlRlk5Sm5JcXV2VHFLWDdSN3lSc3VGZWluOW40SGlEQnNiTUVn?=
 =?utf-8?B?RkYxbFEzU0Z3WHRKRzBZZjhvQVpQcjQ3RTBNT2NrbTJBUHQyckZtOTlVOHVW?=
 =?utf-8?B?bGlLYW1LZ2t5RVFNamFPdEdkVi94aGdQVHdXb0t3WkZLdTZjR3BsMm5xQ1RB?=
 =?utf-8?B?UURBMUNGQXJXL0J3QlRwSUJmdURkd0VCdnNGdnpvdHZFQTJYUVBTbFRkZW5r?=
 =?utf-8?B?c2ppVHZXVGlvQWdGb0dQbnkwNGxwcy9ZM3RaSlRYSEQzWURNQ3EzcGlwbTRP?=
 =?utf-8?B?akZPc0UwcDBNMWx3MVhZUDBBN0VVeGFrUFMxTVVLS3lpWEs5c1RoN1ZVN0xF?=
 =?utf-8?B?U0MzSkZab3NyNTZ2cHJQVSswcXNpSjFlRGE4ZjcwbUZQc21IWUlHSkNqVmdK?=
 =?utf-8?B?aklWY2o0Vlh2UmZEcUNLMThyZDVjWlNpaFpUMFpkWEp1ZVRJOWZPcnpDd2pz?=
 =?utf-8?B?WGRmSFFOZk5pVTdRNmQ5dEpDZzZVVzZVQUFISWpUSERwZWxkZ3hzT3NwUEdB?=
 =?utf-8?B?cHptOWloUndlMlVxV1BxSWxJc2FUZDdQQzR5UTJDcE1HZHNCWi9IQStOdG44?=
 =?utf-8?B?QWM3ZmNJMWc2cGV5aTFxNTQrQ0tNK1FBWFpWS3FxblBwcEFsVUdxSHBPRG50?=
 =?utf-8?B?bHhyQ09xSWNPWHp5dWFoc0hBRUx5TXpPK2pwSnBMS0dUVmVTMnV2VlFwdmNx?=
 =?utf-8?B?cjV2M0p5REJ3ZFBjMGxKbVJMc3ExdXYxUStiWWRaeHhqczlsRG9wakRkbWtP?=
 =?utf-8?B?SkFaR0dqSnJUelRDZTRIeW81bytpcVozVlZZTk5tTm5zWTA0dGxvUEQ0MUdI?=
 =?utf-8?B?cnBDOFplTm90UXhabzA4QlNrUHhESmNreHJsMmVnazIra3pUYzEreXJzbmIz?=
 =?utf-8?B?a0RyOThEeTYzRDQ4MmY2VUs2S1RrK0gwcFdWLzd3eGJqNHFkTjRreUNqQmJh?=
 =?utf-8?B?c2hTOFRtSE9NaEhZVXNnbDBkcHNmVU5lTFU3REVIRVRHQVM3WkhqL3E0MXUr?=
 =?utf-8?B?eERtSVdHQlVveW01Tk1VMkxiZ1d6NmxuSzcwQnBRdE9RUFcyVzlqaFR1cThB?=
 =?utf-8?B?SVNaT2h3WEZzbWdJaW5HOWVGT3dGb0E2aXRZS20yZE5GamFIbTZNNmE3bkFx?=
 =?utf-8?B?cnlnTW9PVXhqMi9yL3RQMXJHWUJRcmg1K2YvQVMwYm9Ea3k4VUI2NTg3d0tU?=
 =?utf-8?B?WEdySHpMTzhMcWRUbGRwV3RFWTl0RExhMmphUGgwZFhIYkhiWlZUbVFKNmZw?=
 =?utf-8?B?MkxFMjZuTldROWI2SG9KOWpBUWNpSWNZTm1tNjkrdVM3NUp3UDh5T0NoRlhP?=
 =?utf-8?B?T1dQSmRJcWswTkZmazh6V21pYTVmZ0ZnbjJsOW83UHltSkJWN0RhTXh6ZHVj?=
 =?utf-8?B?bSswSXBCTndKS2MrSnhmUXlVVzJkNnNkNi9PMmdGejBrRzI2OHJ0Z2g4TC9v?=
 =?utf-8?B?OFVpWVR0VzdUazQ0TVBGRzdhN3c1Y1JCa2tGaDZEd1kwb2hHMC9WWmUxTDBy?=
 =?utf-8?B?NzB4NTBJMTRTNjNPMEpmTnQrakFod1BLem1EMDIyK0laRXFJcjg5RmdzS1dk?=
 =?utf-8?B?SFJlOG1QWjlFUFJBMEF4ckU1cFE0Vm5kOVJCTlZkakVYanB3WVhlRVY1d0p0?=
 =?utf-8?B?Y3NFS2xJSU9GMTdzeklFR2h5RGZlZ1BaSXc0cU12M3JYSmc2clkwelMvVDcz?=
 =?utf-8?Q?fSnGQJhnLYzKWuA4=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 01c1ea13-c64c-4462-c657-08da31ffabaf
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2022 21:05:39.5691
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RD8poBMlJyElIBuWBmZGumJjKljODrxnAo3zer7ynEe5HNCzvFXxIENgH9xXaffkZoOYue7p0WTTSN+gDN5Hcg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR10MB6048
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858
 definitions=2022-05-09_05:2022-05-09,2022-05-09 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0
 suspectscore=0 phishscore=0 spamscore=0 bulkscore=0 mlxscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2202240000 definitions=main-2205090107
X-Proofpoint-GUID: eYfmCoUkIXycAZsh9vTo0jIKHCg5fZkW
X-Proofpoint-ORIG-GUID: eYfmCoUkIXycAZsh9vTo0jIKHCg5fZkW
X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 5/8/22 02:36, Baolin Wang wrote:
> On some architectures (like ARM64), it can support CONT-PTE/PMD size
> hugetlb, which means it can support not only PMD/PUD size hugetlb:
> 2M and 1G, but also CONT-PTE/PMD size: 64K and 32M if a 4K page
> size specified.
> 
> When migrating a hugetlb page, we will get the relevant page table
> entry by huge_pte_offset() only once to nuke it and remap it with
> a migration pte entry. This is correct for PMD or PUD size hugetlb,
> since they always contain only one pmd entry or pud entry in the
> page table.
> 
> However this is incorrect for CONT-PTE and CONT-PMD size hugetlb,
> since they can contain several continuous pte or pmd entry with
> same page table attributes. So we will nuke or remap only one pte
> or pmd entry for this CONT-PTE/PMD size hugetlb page, which is
> not expected for hugetlb migration. The problem is we can still
> continue to modify the subpages' data of a hugetlb page during
> migrating a hugetlb page, which can cause a serious data consistent
> issue, since we did not nuke the page table entry and set a
> migration pte for the subpages of a hugetlb page.
> 
> To fix this issue, we should change to use huge_ptep_clear_flush()
> to nuke a hugetlb page table, and remap it with set_huge_pte_at()
> and set_huge_swap_pte_at() when migrating a hugetlb page, which
> already considered the CONT-PTE or CONT-PMD size hugetlb.
> 
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
>  mm/rmap.c | 24 ++++++++++++++++++------
>  1 file changed, 18 insertions(+), 6 deletions(-)

With the addition of !CONFIG_HUGETLB_PAGE stubs,

Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>
-- 
Mike Kravetz