Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     linux-mm@kvack.org, linux-kernel@vger.kernel.org
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Mike Kravetz <mike.kravetz@oracle.com>, stable@vger.kernel.org
Subject: [PATCH] hugetlb: fix huge_pmd_unshare address update
Date:   Tue, 24 May 2022 13:50:03 -0700
Message-Id: <20220524205003.126184-1-mike.kravetz@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?24NM6F7GljyWyM1yVJC4Y9bAynT9YuyQ08tW1ei16ZWckePZsJeo4JVoW+rQ?=
 =?us-ascii?Q?HQ3IFyx1tq5ZNkIgzo/6WMnwmCMuRWYK1pevXl1u2b38aPdKqmxDebx/GLi3?=
 =?us-ascii?Q?nnJNyFIq3m7vo1iCel+r+1EDhDJF7YZHMI2k166olwq3emaKJGrvaGLSV9yV?=
 =?us-ascii?Q?SYS9/XdpQJb+ZHxZPXjm2weMYzrdChHioAUGepIu6LLQqZxzpsp0xyu4deLj?=
 =?us-ascii?Q?BCtUB07VHfqeBg/xBV6li+wN3Z/rl1eAMSx6rZIIroNE3dwiUrHtN+FsyRcb?=
 =?us-ascii?Q?vMTdntgSYQ30erFoNMfoH8fNrqTmlxKSrYf1CTGRhXDSCkWR8T0nxLV0OP+F?=
 =?us-ascii?Q?QMRVeuudB3j02JoJtNmdoSME6UGwpxQbK3ugcld9/wuYDn//OfG36g3uTIpm?=
 =?us-ascii?Q?5VvQfmrll47F5aVzv1j44rpSPQ0tw5t1Ujd154/7LlSjrjFEnhzN42bjOn2C?=
 =?us-ascii?Q?lcFSWYn6b9bWnGYmbecIr0fAyNnsmI5KnK2lPvh7Zoo5cwwi94Zu1KSYaMgN?=
 =?us-ascii?Q?eBsoa5YAy95UeP4TWD/0enMNmKgDGQ/6GPgB9LzhDEyxNvq1No5BvMlM7PzR?=
 =?us-ascii?Q?AxOSNyg+dg6p5NKRu1v+UI9h+TByjujxfh1TVb0IzqsDeHGnLLSRj7bEf6Lz?=
 =?us-ascii?Q?bTO6F2RekkZajwyOWhwijSDPARrQpJA6gh/cpjGca74ww/mT6mF8e8D3kuPb?=
 =?us-ascii?Q?t5dyBvnEuyalyMkIMimQf+zd5vBj14kNWL6ah9pq8MNWMB/uIiVuX3qtRIGK?=
 =?us-ascii?Q?Lev9KiuZIbBWN6LkhSpwN1M1vf+yjBvCS3jfZaZRTvyEmQZ5K6bewo24PRrf?=
 =?us-ascii?Q?h4L8Znn9pJ2lpf0dMTC1P7iWfLA4xijeaaQ+8OkHwNiRlvASy7jeAfjXcdYN?=
 =?us-ascii?Q?KmZT2RLdqNNDUJJ/cFeNJI2Zr/dKQo7LYSD7C0ruyl1FYQd0xIojvJ8rnNx+?=
 =?us-ascii?Q?6ZsHfMdO4sep/YMVe7oR+vTpQ76Ty8Y45H8RC44FzYMMx3szYJ0W02iowmMI?=
 =?us-ascii?Q?vp7r4EWCWUMZWx9z4czIbadvClEeG5r+edd070ClQIciVhV5FVnOndPMU6Ol?=
 =?us-ascii?Q?OQF54IXdkTCnAj4CEOEV8ep9TJzRVN/tJQPerAGh7XuM3jePhkYXGhDjsNZu?=
 =?us-ascii?Q?UU4eE37hjVGTa+ECMNgUhgNgtzDaDMphsoit9K6+jxskA9iAm3sA35in835b?=
 =?us-ascii?Q?9UeHqwyaIiTNKr6/1xZxKp7HDsw5MEiDsrU/CgTYQ3AzJeEVL9uvY/IW2Xuo?=
 =?us-ascii?Q?kwjh4N2tM1tWCSDM7NJtijiQ6HFSVF/azyeW0gwk1toahAwD4PdiK0GhEt+k?=
 =?us-ascii?Q?Pnkn69sNt1hRdPIj8esOv3zAFH5pwy2E9slOnj8vPePLt84q0zdPThj2p3Nd?=
 =?us-ascii?Q?FGYY/Ay+4G+C9bGvWTb77Uxx4ttIBF+DFVGFHisNhI8791p5Z7SyUWkOdI6F?=
 =?us-ascii?Q?LErYPDFEmW8bPeTXmFRoJ721j3wNYGpzjzJ3dFUJ2FEQ5WhY4f8JlSfOxuI8?=
 =?us-ascii?Q?UbWa75SHt5ViR5glhM1J/sq6Y3WchMXNzZsyqtETocI2ngVGGRjNUcJyRbXq?=
 =?us-ascii?Q?8npgraagTXpfGP9OFZt9GNa919Bvbflr9vnsq8/I2+aofsmMwgBcukLLoBC9?=
 =?us-ascii?Q?uKdF69uMMzvHfucdGOiwefE12oE9C7hfF7J7BUz/vJxvBGxn8QSas8Gg4mFq?=
 =?us-ascii?Q?+uzpQy7X9xkHv1AkUNh599tlN5myoTDDpfgEwDKL4nOkmIy4D5gG2RwS6YpS?=
 =?us-ascii?Q?MfJLbFweRN8xHqul1PMGxcGf+OXqc40=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5be55647-7911-4a82-d387-08da3dc70c06
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2022 20:50:33.9827
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: w3qtrAidsrr2V8OoHPtJti6yoPjDVchHRilpkyyZuzALsZ0V0UGBPask6K021/oGVXoSKDEt0nkdTDNcNUuxVw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5618
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.874
 definitions=2022-05-24_08:2022-05-23,2022-05-24 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0
 suspectscore=0 mlxlogscore=820 adultscore=0 spamscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000
 definitions=main-2205240102
X-Proofpoint-ORIG-GUID: 5l9193N3t5EELc0hV_JtXyCrDR8iqoBQ
X-Proofpoint-GUID: 5l9193N3t5EELc0hV_JtXyCrDR8iqoBQ
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

The routine huge_pmd_unshare is passed a pointer to an address
associated with an area which may be unshared.  If unshare is successful
this address is updated to 'optimize' callers iterating over huge page
addresses.  For the optimization to work correctly, address should be
updated to the last huge page in the unmapped/unshared area.  However,
in the common case where the passed address is PUD_SIZE aligned, the
address is incorrectly updated to the address of the preceding huge
page.  That wastes CPU cycles as the unmapped/unshared range is scanned
twice.

Cc: <stable@vger.kernel.org>
Fixes: 39dde65c9940 ("shared page table for hugetlb page")
Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
---
 mm/hugetlb.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 01f0e2e5ab48..7c468ac1d069 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -6755,7 +6755,14 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma,
 	pud_clear(pud);
 	put_page(virt_to_page(ptep));
 	mm_dec_nr_pmds(mm);
-	*addr = ALIGN(*addr, HPAGE_SIZE * PTRS_PER_PTE) - HPAGE_SIZE;
+	/*
+	 * This update of passed address optimizes loops sequentially
+	 * processing addresses in increments of huge page size (PMD_SIZE
+	 * in this case).  By clearing the pud, a PUD_SIZE area is unmapped.
+	 * Update address to the 'last page' in the cleared area so that
+	 * calling loop can move to first page past this area.
+	 */
+	*addr |= PUD_SIZE - PMD_SIZE;
 	return 1;
 }
 
-- 
2.35.3