Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Liam Howlett <liam.howlett@oracle.com>
To:     "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>
CC:     Liam Howlett <liam.howlett@oracle.com>,
        Liam Howlett <liam.howlett@oracle.com>
Subject: [PATCH v2 16/44] mlock: Convert mlock to vma iterator
Thread-Topic: [PATCH v2 16/44] mlock: Convert mlock to vma iterator
Thread-Index: AQHZITojRp+rlyDyuU+9GjNxV6YrOA==
Date:   Thu, 5 Jan 2023 19:15:57 +0000
Message-ID: <20230105191517.3099082-17-Liam.Howlett@oracle.com>
References: <20230105191517.3099082-1-Liam.Howlett@oracle.com>
In-Reply-To: <20230105191517.3099082-1-Liam.Howlett@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?TK8QCv5iWxa+J24swxMgongjgcVnPAc/z3dCdaJPK+85QOCT9aPNiz8bd9?=
 =?iso-8859-1?Q?Kv6IAPMGri43tYt1+uzWQCtxrFIJDr3dqAZDfo6ynUtQpSSj1B6hs0BrH+?=
 =?iso-8859-1?Q?ljVmrK/QA6fqMdB8sLlGa+5foFMHfpBzwvjpCdETzKUjhwvV9Xowjo9UP2?=
 =?iso-8859-1?Q?qzMbxVp10MJC6MXf8NswKPUR9js3eZLHL24R9Tl2FELU5lfV7piPqKIQHS?=
 =?iso-8859-1?Q?verS6TxnenJG/cMtz4tSnvkHmNb6Q15ZURhk7yaU8tcN+rJZgktyjHCQAR?=
 =?iso-8859-1?Q?VOnLWefHHcIzI6S6hYut+koPP4I+VFLM3Kvdrr29Qls2Cxgyjh89eKbzNL?=
 =?iso-8859-1?Q?WOTxb8vqGpCjLY2esZ/WgSApeX+lwzRLq+CQYgIB8J0ay/dFmgjWvVGEig?=
 =?iso-8859-1?Q?Ph+E2C3QLMCaZll9/hamTy1e/GG2ycIgyNET7DS5RQed9sYI3j9sPST2f7?=
 =?iso-8859-1?Q?YK/7uI4J5BetsOfxCUObkCqGmLPzH1HZpXPyV/VzgmifXk+2v5gRwJOFYw?=
 =?iso-8859-1?Q?r15IeHWw7Xox8SecHgE+btaMlqGsNh7dyGpmk2Jwd97eY8iJ7J00pnx4Yr?=
 =?iso-8859-1?Q?jazpjRsYzdU26G8OWR0OHz8DFxY6SvKfbHecf/2xOMi2+Nf06CSAsq4Kjm?=
 =?iso-8859-1?Q?+43q3dNbmZVhaa27wgY5lm+aCq+sgAn6sOqwGg1eQkruqhZ3SzCYi+iMOw?=
 =?iso-8859-1?Q?nZMW2HnfIcGa14dq0xbKgdYB/WX9g7SXj9mZAOr5yvmoO2VqmyI5S63alr?=
 =?iso-8859-1?Q?nM9nF2w518peF2sVDTYtmUSG9XIF1EZPBmojj90G/rcgFZREWwLFczjkk3?=
 =?iso-8859-1?Q?fMb4o51i18RcS12gEJEhxwhuNazFMK0kciXRuBYwAVJDHhpx1o5nrzmBYF?=
 =?iso-8859-1?Q?Tljuyls5zhKGssMgiOUUShDFhWIirl5KMKFHUO6g9ky+aEc2FwePE0B1v8?=
 =?iso-8859-1?Q?q/4trSya/IcvcInlBnmeiBVmhtpVLB1ZZLhGlC1Si+4jJ+OecXq4jWmSaq?=
 =?iso-8859-1?Q?HHySUn0U0QvPW5CzQrnmNkPrkQa+2e2bRWXgEnUNVUOMXY9W+v1uNOAOul?=
 =?iso-8859-1?Q?1SpOypJupSgWpn37q9CuYFSIkAjdlY4rSKZxfbu3uIvPWhk2byp7WL0d5P?=
 =?iso-8859-1?Q?wlY9doSTxP1v2fAC/cmtRCVCZR/m2bckwsr1ZF/pSM9zzCouiRj03K8Tte?=
 =?iso-8859-1?Q?BgTeMkQADvSNBu8iZoNknnVHoJpWlD0Xv4rAVfgh5txDQ/GlmXMbaMIDwD?=
 =?iso-8859-1?Q?IPDFcZ4hWrbs5lRuqnNfELgDJZw7UxPVSjt17UhvJC8Vog9HNS98tV0HZb?=
 =?iso-8859-1?Q?llvEmlys7pm1WJDT5/134qnVFx+3tdeu/PpQyrxyEtMtF/PpuvEO4Avffp?=
 =?iso-8859-1?Q?OKRUhlkE/n+vU+w2E0VcPEbIBNm4g0Zq3HxA66t2YUQETN71JQeKTS4hWe?=
 =?iso-8859-1?Q?KiEu/GwAG0rTZRO25+JVKV+pJapctry/8Fb12H5zqcnLW0RlQHZeIq2hWy?=
 =?iso-8859-1?Q?O8Bl4o7BFIFluwliU4RQ28IxmuAFdIV9Z51lRlWYf4LieqMIYToQIgrmP+?=
 =?iso-8859-1?Q?vxHR2IWMHak5gN6hX/Kfy6I2ReGvpZjmBb5xJDCxb4cSt1T3cJ79X3Lt65?=
 =?iso-8859-1?Q?9EYfjR6xRlisHag9OlawukElEi35C4t0qma5RBUmlqvkG+HTlr+o72TQ?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: EgoUwO6bDm1yd6r6ifB1PFJDmfRDF+RQeKboOr50edALgHdOfxi1qaY4+3uQRsa/NlNZ7hMPL3TNTkD0/WPx2oHDaLUSVSbdzQclOPPW8uhbX8+MOpL7cvqEX+pXHNlHKEcn8S4OHqTRJeVHkFBvBIbg1wbJF31um/lbJ/rO9dcZrPhtMKK4DalGi66gxiitfwLQLurXXhEaY364V3H2tA3wR2zdqMlsBZ7H/slBU+Mwj+i389gAF8f3fmTQbj+kiR5M9x5pLife0L649uf+jfLqcr7ZylyUURM/9zvVYgbYS3JCfWtUCfYM2pWbk9ykGF1r7QXuGzhTelbwpfU10jjix4V6ilIyHCvkKlvzDaSNhQ/V2fQQlTh113J6p83zR1hpUvjQZfYFhP5Zi2GcJLGCQb0u5OU9eMm2NipdVEfDJsePUqJlVP2WCkEyj1Ui67HGDiQ4Nx2JvgA5TFtHJO0qm7ynpBMIWPiXwfZddWOKHR3c3Fj/aIyn6FWTeD/q3fYoRw0uJL6Gyi8PAyYriG/rnKYL0WEcWPGqIdtZbF3JRpvasVcFD3Uuojl60YZ5SmgNZVnHP6OWfE4ertlNUG9b6DGRdiJXvLMMaOVngStvz5Wgki9bDYjEUs+DuZP7GnQ9KpOM0IvSzzl2Rgiicj3ONp+6MmSf4AqJsLLWbL2pEQXzLSP0nVHaFWp5oCbSYmFsF6ksK1Ij0UWFNeqFtmofTBHnfcInC3mLNi2WUxJLY9NHifgAwNdPXOGbawJ/oHI1NeUJ1Lxhf6GByEOC0s8Nvf2LbrhNXFM07/DrrMInvc2GplL08WUx2GbSh7U9mfmOiBM4+9EfmcDtQs6SnIqwbOCmNRGqW5pBNZYcnQvAR5bHyX1GFASC3rgo6w2SmSLRlRhlp5pTG5DOKnABIw==
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 85dca05d-b3f7-463a-10ca-08daef514812
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Jan 2023 19:15:57.3444
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: zDfNGMIb2+N+MeQjqFdw2+jwMU7InTBozQDJuOcmv2hMs5mv0a+KMHdtrJGMzKs5nzcnwUuAIPCmExDu0WOZzQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR10MB6075
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2023-01-05_10,2023-01-05_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 suspectscore=0
 mlxlogscore=999 spamscore=0 bulkscore=0 phishscore=0 mlxscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2212070000 definitions=main-2301050152
X-Proofpoint-ORIG-GUID: syyomKtQtid0ioaNdMUt3xTi5XW06ZtO
X-Proofpoint-GUID: syyomKtQtid0ioaNdMUt3xTi5XW06ZtO
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: "Liam R. Howlett" <Liam.Howlett@Oracle.com>

Use the vma iterator so that the iterator can be invalidated or updated
to avoid each caller doing so.

Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
---
 mm/mlock.c | 57 +++++++++++++++++++++++++++---------------------------
 1 file changed, 28 insertions(+), 29 deletions(-)

diff --git a/mm/mlock.c b/mm/mlock.c
index 7032f6dd0ce1..f06b02b631b5 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -401,8 +401,9 @@ static void mlock_vma_pages_range(struct vm_area_struct=
 *vma,
  *
  * For vmas that pass the filters, merge/split as appropriate.
  */
-static int mlock_fixup(struct vm_area_struct *vma, struct vm_area_struct *=
*prev,
-	unsigned long start, unsigned long end, vm_flags_t newflags)
+static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vm=
a,
+	       struct vm_area_struct **prev, unsigned long start,
+	       unsigned long end, vm_flags_t newflags)
 {
 	struct mm_struct *mm =3D vma->vm_mm;
 	pgoff_t pgoff;
@@ -417,22 +418,22 @@ static int mlock_fixup(struct vm_area_struct *vma, st=
ruct vm_area_struct **prev,
 		goto out;
=20
 	pgoff =3D vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT);
-	*prev =3D vma_merge(mm, *prev, start, end, newflags, vma->anon_vma,
-			  vma->vm_file, pgoff, vma_policy(vma),
-			  vma->vm_userfaultfd_ctx, anon_vma_name(vma));
+	*prev =3D vmi_vma_merge(vmi, mm, *prev, start, end, newflags,
+			vma->anon_vma, vma->vm_file, pgoff, vma_policy(vma),
+			vma->vm_userfaultfd_ctx, anon_vma_name(vma));
 	if (*prev) {
 		vma =3D *prev;
 		goto success;
 	}
=20
 	if (start !=3D vma->vm_start) {
-		ret =3D split_vma(mm, vma, start, 1);
+		ret =3D vmi_split_vma(vmi, mm, vma, start, 1);
 		if (ret)
 			goto out;
 	}
=20
 	if (end !=3D vma->vm_end) {
-		ret =3D split_vma(mm, vma, end, 0);
+		ret =3D vmi_split_vma(vmi, mm, vma, end, 0);
 		if (ret)
 			goto out;
 	}
@@ -471,7 +472,7 @@ static int apply_vma_lock_flags(unsigned long start, si=
ze_t len,
 	unsigned long nstart, end, tmp;
 	struct vm_area_struct *vma, *prev;
 	int error;
-	MA_STATE(mas, &current->mm->mm_mt, start, start);
+	VMA_ITERATOR(vmi, current->mm, start);
=20
 	VM_BUG_ON(offset_in_page(start));
 	VM_BUG_ON(len !=3D PAGE_ALIGN(len));
@@ -480,39 +481,37 @@ static int apply_vma_lock_flags(unsigned long start, =
size_t len,
 		return -EINVAL;
 	if (end =3D=3D start)
 		return 0;
-	vma =3D mas_walk(&mas);
+	vma =3D vma_find(&vmi, end);
 	if (!vma)
 		return -ENOMEM;
=20
+	prev =3D vma_prev(&vmi);
 	if (start > vma->vm_start)
 		prev =3D vma;
-	else
-		prev =3D mas_prev(&mas, 0);
=20
-	for (nstart =3D start ; ; ) {
-		vm_flags_t newflags =3D vma->vm_flags & VM_LOCKED_CLEAR_MASK;
+	nstart =3D start;
+	tmp =3D vma->vm_start;
+	for_each_vma_range(vmi, vma, end) {
+		vm_flags_t newflags;
=20
-		newflags |=3D flags;
+		if (vma->vm_start !=3D tmp)
+			return -ENOMEM;
=20
+		newflags =3D vma->vm_flags & VM_LOCKED_CLEAR_MASK;
+		newflags |=3D flags;
 		/* Here we know that  vma->vm_start <=3D nstart < vma->vm_end. */
 		tmp =3D vma->vm_end;
 		if (tmp > end)
 			tmp =3D end;
-		error =3D mlock_fixup(vma, &prev, nstart, tmp, newflags);
+		error =3D mlock_fixup(&vmi, vma, &prev, nstart, tmp, newflags);
 		if (error)
 			break;
 		nstart =3D tmp;
-		if (nstart < prev->vm_end)
-			nstart =3D prev->vm_end;
-		if (nstart >=3D end)
-			break;
-
-		vma =3D find_vma(prev->vm_mm, prev->vm_end);
-		if (!vma || vma->vm_start !=3D nstart) {
-			error =3D -ENOMEM;
-			break;
-		}
 	}
+
+	if (vma_iter_end(&vmi) < end)
+		return -ENOMEM;
+
 	return error;
 }
=20
@@ -658,7 +657,7 @@ SYSCALL_DEFINE2(munlock, unsigned long, start, size_t, =
len)
  */
 static int apply_mlockall_flags(int flags)
 {
-	MA_STATE(mas, &current->mm->mm_mt, 0, 0);
+	VMA_ITERATOR(vmi, current->mm, 0);
 	struct vm_area_struct *vma, *prev =3D NULL;
 	vm_flags_t to_add =3D 0;
=20
@@ -679,15 +678,15 @@ static int apply_mlockall_flags(int flags)
 			to_add |=3D VM_LOCKONFAULT;
 	}
=20
-	mas_for_each(&mas, vma, ULONG_MAX) {
+	for_each_vma(vmi, vma) {
 		vm_flags_t newflags;
=20
 		newflags =3D vma->vm_flags & VM_LOCKED_CLEAR_MASK;
 		newflags |=3D to_add;
=20
 		/* Ignore errors */
-		mlock_fixup(vma, &prev, vma->vm_start, vma->vm_end, newflags);
-		mas_pause(&mas);
+		mlock_fixup(&vmi, vma, &prev, vma->vm_start, vma->vm_end,
+			    newflags);
 		cond_resched();
 	}
 out:
--=20
2.35.1