Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Liam Howlett <liam.howlett@oracle.com>
To:     "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Hugh Dickins <hughd@google.com>
CC:     Yu Zhao <yuzhao@google.com>
Subject: [PATCH v12 52/69] mm/khugepaged: stop using vma linked list
Thread-Topic: [PATCH v12 52/69] mm/khugepaged: stop using vma linked list
Thread-Index: AQHYm97uCPAR09mT5E6gqFxpLKbjfQ==
Date:   Wed, 20 Jul 2022 02:18:00 +0000
Message-ID: <20220720021727.17018-53-Liam.Howlett@oracle.com>
References: <20220720021727.17018-1-Liam.Howlett@oracle.com>
In-Reply-To: <20220720021727.17018-1-Liam.Howlett@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?qNp71GGBMz7byvKAi83siM8Hj6TLL+ARZnDde0kaDYj87skb2D4MbEA3wV?=
 =?iso-8859-1?Q?Z4zedQUoTRmJOMaLxW202M/tGYkz6YQMzkhx3+vVu2/2GQ8AIPDLkNQxw7?=
 =?iso-8859-1?Q?YNDSZidU7POcKPf8mM2lA2T/VjO6HEto9BHstfiEZcuHs5XSfTsLGt9l+L?=
 =?iso-8859-1?Q?fbg+A87npd1UObtgyaxBydCGotEBxJuY0GQp9u/7p4NOG/iOCDyed91rM4?=
 =?iso-8859-1?Q?Sms0eXKg5KE3UzURtaLLSWtm5ut3gY8ZokFqCKHP2HgXUYVGGCxmhmCe1c?=
 =?iso-8859-1?Q?vUFP5uFTSdTkdp1tw3zT17yy3KfuNXgnehMoi2OnciuZ+cDZL0LqJ4hsy1?=
 =?iso-8859-1?Q?GM2gG4aCrnUE19K3HSokxRutkTyPFzo3EktojY1UIgTnqtFnmQ3cn0I7oE?=
 =?iso-8859-1?Q?LTbGWn+DWgXJgHzjOk0KnSh1Y8n3DZqE7YC2zuFJjktiFD/OsS6zhDk8NO?=
 =?iso-8859-1?Q?I6dNURuwyEKkwEOx3XiLhjCvcIwn8SaUVUuha+QVwYPxSjl6o/nRPSk9DX?=
 =?iso-8859-1?Q?PGyzjybtYJSARRHrMP9S3g58dTiBfWSQB05HD1oKygxF80Yl2bjnTw4/hd?=
 =?iso-8859-1?Q?kBCn8SMuzczYyA+dW9qxBYNdNexchxsuwUKcfelRvrUMaR2IS+glX8UQP4?=
 =?iso-8859-1?Q?L017GHD/BLvPTu42dnLimTipma+yRjNEjo1X5d/UdgNfLyQDUX2tle2ZZ/?=
 =?iso-8859-1?Q?nPoD0nXPiFU+vujAFh7m/cYuYFegU5kPRWA8I/xw7uuNduazGa+YhpIho6?=
 =?iso-8859-1?Q?sjnXwF3vXNNGBxi6OGC8nvCBfXbgJCDjdx5gpnWen0RJWYE9U06yfeB0gX?=
 =?iso-8859-1?Q?chZ+PGiOwfvaJyf+/2gDnYFaKUAaAPUjkzIVkEQACBpLbTzXjqPyptV2Sx?=
 =?iso-8859-1?Q?VbhhL6BOpfPQC7r615hqt9deINIQKqZWFW8J0kcgz2rsGyIYbVt/KEbPQa?=
 =?iso-8859-1?Q?VzyWkPOo/Gr5BXBkPWsDRs+f1r8s2XXWnn2bMPE/qCxhn1BKcPTDcQZGA9?=
 =?iso-8859-1?Q?DQCesNkqUpz+8bl98g/wFDZwJf3lpnj5ya+RYdjYabYCvrofoR/wSuh0DM?=
 =?iso-8859-1?Q?zKSZRiodNXKs5/D4432dx0CYuU8CC/1Ao6M+pLARFcZ/HUaqTKPrp3VEtl?=
 =?iso-8859-1?Q?CqWOlqYHhvhRKNDgorQ3Kc5m3754Yz6WhChctfZn+zcEnuOJCabMlA/DYS?=
 =?iso-8859-1?Q?bmF7q/it/X1e/PZ3OLri3iFm/HfjpZt4S4cANpSkt0MvY7uDVju8M1awHt?=
 =?iso-8859-1?Q?9soid7MYZxlcFKdeuoaNb0fCr/xCSD2OqAlD2sTAHb9j9Y2Ybc92m5Q1QM?=
 =?iso-8859-1?Q?q+iheYUxJku6OWML3xQ4X51s+P74DyVPCA2ogzuffwZ71xlALfPz8Sh8RW?=
 =?iso-8859-1?Q?e4VpNsn3TyjUNHhs0G4F5mLpX8QCMySFLSXjdbqF8ALcJkZ79X5K1cLrQ+?=
 =?iso-8859-1?Q?kukxRs/FHgS9vEl6ojW6C2C3r3hWTEqZWxAPHhCjl99KvK5D6cdqQEHSWX?=
 =?iso-8859-1?Q?LMAJKHi9osCP0WOunho+jwfvl4tybo25NJpDaVKqSmubhWVomEVmocJcNb?=
 =?iso-8859-1?Q?qYuKM9BE0dGypR07Ytn4C2pcLcQ2ClksErdjc7EXfYHFkxCiyTYrzelWby?=
 =?iso-8859-1?Q?+WFhZYRp8WvubHGJCs4/56JVUQjdAio/CmbUpnZb9RfH0rtjaNl0Rzgg?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3321d93f-07a4-4602-a4d1-08da69f6299f
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jul 2022 02:18:00.1957
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: nqm5PIyNn8SRwU1bj7SbkiEkRudCsguepdWtkJ1BdZ41hcjEcmZAv3NeBnVSLu0xtHVSlTH1dGdzfODLFVsLdg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR10MB3797
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1
 definitions=2022-07-19_10,2022-07-19_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0
 mlxlogscore=999 malwarescore=0 adultscore=0 mlxscore=0 spamscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2206140000 definitions=main-2207200008
X-Proofpoint-GUID: 5cimcP_UWSe6v0Ydn5b1ciGakUQohXR3
X-Proofpoint-ORIG-GUID: 5cimcP_UWSe6v0Ydn5b1ciGakUQohXR3
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: "Matthew Wilcox (Oracle)" <willy@infradead.org>

Use vma iterator & find_vma() instead of vma linked list.

Link: https://lkml.kernel.org/r/20220504011345.662299-37-Liam.Howlett@oracl=
e.com
Link: https://lkml.kernel.org/r/20220621204632.3370049-53-Liam.Howlett@orac=
le.com
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Howells <dhowells@redhat.com>
Cc: SeongJae Park <sj@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
 mm/huge_memory.c |  4 ++--
 mm/khugepaged.c  | 11 ++++++++---
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index f7248002dad9..f44ffd3bbfae 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2266,11 +2266,11 @@ void vma_adjust_trans_huge(struct vm_area_struct *v=
ma,
 	split_huge_pmd_if_needed(vma, end);
=20
 	/*
-	 * If we're also updating the vma->vm_next->vm_start,
+	 * If we're also updating the next vma vm_start,
 	 * check if we need to split it.
 	 */
 	if (adjust_next > 0) {
-		struct vm_area_struct *next =3D vma->vm_next;
+		struct vm_area_struct *next =3D find_vma(vma->vm_mm, vma->vm_end);
 		unsigned long nstart =3D next->vm_start;
 		nstart +=3D adjust_next;
 		split_huge_pmd_if_needed(next, nstart);
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 8dbd68c414d9..637bfecd6bf5 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -2092,10 +2092,12 @@ static unsigned int khugepaged_scan_mm_slot(unsigne=
d int pages,
 	__releases(&khugepaged_mm_lock)
 	__acquires(&khugepaged_mm_lock)
 {
+	struct vma_iterator vmi;
 	struct mm_slot *mm_slot;
 	struct mm_struct *mm;
 	struct vm_area_struct *vma;
 	int progress =3D 0;
+	unsigned long address;
=20
 	VM_BUG_ON(!pages);
 	lockdep_assert_held(&khugepaged_mm_lock);
@@ -2119,11 +2121,14 @@ static unsigned int khugepaged_scan_mm_slot(unsigne=
d int pages,
 	vma =3D NULL;
 	if (unlikely(!mmap_read_trylock(mm)))
 		goto breakouterloop_mmap_lock;
-	if (likely(!khugepaged_test_exit(mm)))
-		vma =3D find_vma(mm, khugepaged_scan.address);
=20
 	progress++;
-	for (; vma; vma =3D vma->vm_next) {
+	if (unlikely(khugepaged_test_exit(mm)))
+		goto breakouterloop;
+
+	address =3D khugepaged_scan.address;
+	vma_iter_init(&vmi, mm, address);
+	for_each_vma(vmi, vma) {
 		unsigned long hstart, hend;
=20
 		cond_resched();
--=20
2.35.1