Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Liam Howlett <liam.howlett@oracle.com>
To:     "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        "damon @ lists . linux . dev" <damon@lists.linux.dev>,
        SeongJae Park <sj@kernel.org>,
        David Hildenbrand <david@redhat.com>
Subject: [PATCH v10 57/69] mm/mlock: use vma iterator and maple state instead
 of vma linked list
Thread-Topic: [PATCH v10 57/69] mm/mlock: use vma iterator and maple state
 instead of vma linked list
Thread-Index: AQHYhbAUk4MnBXy2LkmXMJ9TrmCE6A==
Date:   Tue, 21 Jun 2022 20:47:11 +0000
Message-ID: <20220621204632.3370049-58-Liam.Howlett@oracle.com>
References: <20220621204632.3370049-1-Liam.Howlett@oracle.com>
In-Reply-To: <20220621204632.3370049-1-Liam.Howlett@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?8xUEx4Wmv6yb9Emhbr1NXDbGUb2FppyS/GdbqzZdB0FgcyXrimwRWAwOhN?=
 =?iso-8859-1?Q?ZVwXtmxMNi3dNazfUHYPjab3g1C2fPq7ZqotK6aU1MH/BmglVIbXUroxVX?=
 =?iso-8859-1?Q?P3sCZWA62MiGfBQEw4VX7UqGUexPnIA8mT7i3yZbJNU10XPipbVfASCydZ?=
 =?iso-8859-1?Q?Vwa6RLJZoaVf/jJJ8gf6nGHoiU8zYVwtjKTwp1bmkkVQ4FDl9EXTH17Ngc?=
 =?iso-8859-1?Q?CF+KxSSBOHrk5TgaRvNhwOA/qp9b1pmEHNzjA6GWFGMHAcUJFVsnTUfVTl?=
 =?iso-8859-1?Q?TlsTBnZZBmuxyUcL9hry9oz3T9V0Hw7h7wMhKJenfX+EtdduwxNUYB2drg?=
 =?iso-8859-1?Q?Dphp6TLmU5P/6lz/A1ehx8otE++YVLupr2tXSoTfYuMoNj6iXgo1G2wtKL?=
 =?iso-8859-1?Q?kKKucychcNK+4s1d6rvMjQ7ZgSYfe/DW2cLknJWvYHZlXOPO/8WHEvV8Kz?=
 =?iso-8859-1?Q?AEB6segMPA8jhku0+VnFWFAucLobyaXADhJcpHa58DgPdM9SpfzJ3e9Ans?=
 =?iso-8859-1?Q?cwwt5tHvV5gYJQOHJuSqzL+1C1AqvCDhk9GtiDkXZpWgScd1gMUB7thQRr?=
 =?iso-8859-1?Q?+JcZUlN/27gaPKcvtviGY2g/kJsiIKQqv7MPazfHZBO+JBpZLLU/sPkVau?=
 =?iso-8859-1?Q?5F1EElqI8GCncHAO9Jk6rOXDhmqUrUHVoMoNfqRefvDd4jlFxEXoqTy8iE?=
 =?iso-8859-1?Q?q4IJXb9oi2h3bi4kCGjaRBh/VQzSIl838q/2OBOadq91r37pU65DHH6jWl?=
 =?iso-8859-1?Q?Mq1IldEPk5CROPErZDs28BkeiK03LUOYGLZcAXlcHIkknaqNOGUwmZWgDl?=
 =?iso-8859-1?Q?Hwtwl9J5/pMxfVsPM3GgzC+GNBdSktcwrTm5iHt1CYMaLQrxne2Wm95zlh?=
 =?iso-8859-1?Q?YUsIBBaqTtUhUVoSr7mUWgcfWJ1tBb9yfME4NJ5LIhCncow/6kNyNJrgbS?=
 =?iso-8859-1?Q?ZhrTXz8k+5Irh3e+ibVSfiT3safbceyTeN6rat1mOcaexI32P4zS8WruaU?=
 =?iso-8859-1?Q?8leC9S0hMl6e2jrZ3GdVxDq4rtvaptxyNSF+zsv1u+Mnbui+OkZo/WZRRk?=
 =?iso-8859-1?Q?zxZrhbo5MIOviQOYiUZXmNhcBsWUZzm89RZSEvVar4s6ZpSlLVzElBMkyZ?=
 =?iso-8859-1?Q?aiXHNDF2J1e6wCeb3zUH4u9csPG1oMndjPdbcBQ2tygwCogPIXRy9vw8Hj?=
 =?iso-8859-1?Q?KZuJLebBqQTO8pUOQOz1SNpefrWILFx24eBtqOoYeBDmDt5Gl02RrO1aDV?=
 =?iso-8859-1?Q?1L9yWram5x5Sce8we5U5eh95UJXLqK1Xq7tSDiLaJqLNJeGqThOqod2h2E?=
 =?iso-8859-1?Q?8MFl4l6+XEeAU+dDjTVRYdOJFve9sA8GAlBdIIW46zqCcgL18NiAGXhK3K?=
 =?iso-8859-1?Q?zYYhy3T0yGS4sH1fDPDivd8TB2RfKqTKhNg56aeqyxXJxb2Ln+I+rSNJQL?=
 =?iso-8859-1?Q?BBtU5bJtsEsPecr778IIUGY+tViTBHywusnK71ajQ608/+Ny48xJH1Og5M?=
 =?iso-8859-1?Q?XeSBzw2fRhS0EA+rNU4QBDpNcZ9kiF8n3Wjds4jraMXQsDQIpYQqDDi3Nz?=
 =?iso-8859-1?Q?ZeSzUGlcvgk28Q4ZUAWuGUtZph1w4Mece6kJGGDJoawn9+n6ZpvMRHFyAU?=
 =?iso-8859-1?Q?uYYXMgU/e08ZZjNyGooHriU5PPpcHOD6b4AWI2AYXqKkHukMCy165Ihb39?=
 =?iso-8859-1?Q?4KbJYSShufJRDSHBRcarJWFMvaoZPIuAHBgH4rwzf3OhFm8GfuvZqDEI0Z?=
 =?iso-8859-1?Q?G3V1wnfGFcbfq5bKQ8i0XIPnsKCKYmiRl06cD4bQVAi9lXD/NPoem8Fo78?=
 =?iso-8859-1?Q?NXmU/iYzC0dOPfO2WPAhichX3sWlccw=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1db363bc-942d-46c3-9d5f-08da53c75153
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jun 2022 20:47:11.3470
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 7ut+MDznLWlwzAaQDQG7yZ9AiP4tKsVmzscZmm8CnpHen/I7khzJVj/1sNi9dr9T9+HzkmbJb0LxJjOXMb3kiA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB5392
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883
 definitions=2022-06-21_09:2022-06-21,2022-06-21 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0
 mlxlogscore=999 bulkscore=0 phishscore=0 suspectscore=0 spamscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2204290000 definitions=main-2206210087
X-Proofpoint-ORIG-GUID: WptGjsrZoBgEokEwDBGfLROMhAikzx-3
X-Proofpoint-GUID: WptGjsrZoBgEokEwDBGfLROMhAikzx-3
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: "Matthew Wilcox (Oracle)" <willy@infradead.org>

Handle overflow checking in count_mm_mlocked_page_nr() differently.

Link: https://lkml.kernel.org/r/20220504011345.662299-42-Liam.Howlett@oracl=
e.com
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Liam R. Howlett <Liam.Howlett@Oracle.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Howells <dhowells@redhat.com>
Cc: SeongJae Park <sj@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
 mm/mlock.c | 35 +++++++++++++++++++----------------
 1 file changed, 19 insertions(+), 16 deletions(-)

diff --git a/mm/mlock.c b/mm/mlock.c
index 716caf851043..c41604ba5197 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -471,6 +471,7 @@ static int apply_vma_lock_flags(unsigned long start, si=
ze_t len,
 	unsigned long nstart, end, tmp;
 	struct vm_area_struct *vma, *prev;
 	int error;
+	MA_STATE(mas, &current->mm->mm_mt, start, start);
=20
 	VM_BUG_ON(offset_in_page(start));
 	VM_BUG_ON(len !=3D PAGE_ALIGN(len));
@@ -479,13 +480,14 @@ static int apply_vma_lock_flags(unsigned long start, =
size_t len,
 		return -EINVAL;
 	if (end =3D=3D start)
 		return 0;
-	vma =3D find_vma(current->mm, start);
-	if (!vma || vma->vm_start > start)
+	vma =3D mas_walk(&mas);
+	if (!vma)
 		return -ENOMEM;
=20
-	prev =3D vma->vm_prev;
 	if (start > vma->vm_start)
 		prev =3D vma;
+	else
+		prev =3D mas_prev(&mas, 0);
=20
 	for (nstart =3D start ; ; ) {
 		vm_flags_t newflags =3D vma->vm_flags & VM_LOCKED_CLEAR_MASK;
@@ -505,7 +507,7 @@ static int apply_vma_lock_flags(unsigned long start, si=
ze_t len,
 		if (nstart >=3D end)
 			break;
=20
-		vma =3D prev->vm_next;
+		vma =3D find_vma(prev->vm_mm, prev->vm_end);
 		if (!vma || vma->vm_start !=3D nstart) {
 			error =3D -ENOMEM;
 			break;
@@ -526,24 +528,23 @@ static unsigned long count_mm_mlocked_page_nr(struct =
mm_struct *mm,
 {
 	struct vm_area_struct *vma;
 	unsigned long count =3D 0;
+	unsigned long end;
+	VMA_ITERATOR(vmi, mm, start);
=20
 	if (mm =3D=3D NULL)
 		mm =3D current->mm;
=20
-	vma =3D find_vma(mm, start);
-	if (vma =3D=3D NULL)
-		return 0;
-
-	for (; vma ; vma =3D vma->vm_next) {
-		if (start >=3D vma->vm_end)
-			continue;
-		if (start + len <=3D  vma->vm_start)
-			break;
+	/* Don't overflow past ULONG_MAX */
+	if (unlikely(ULONG_MAX - len < start))
+		end =3D ULONG_MAX;
+	else
+		end =3D start + len;
+	for_each_vma_range(vmi, vma, end) {
 		if (vma->vm_flags & VM_LOCKED) {
 			if (start > vma->vm_start)
 				count -=3D (start - vma->vm_start);
-			if (start + len < vma->vm_end) {
-				count +=3D start + len - vma->vm_start;
+			if (end < vma->vm_end) {
+				count +=3D end - vma->vm_start;
 				break;
 			}
 			count +=3D vma->vm_end - vma->vm_start;
@@ -659,6 +660,7 @@ SYSCALL_DEFINE2(munlock, unsigned long, start, size_t, =
len)
  */
 static int apply_mlockall_flags(int flags)
 {
+	MA_STATE(mas, &current->mm->mm_mt, 0, 0);
 	struct vm_area_struct *vma, *prev =3D NULL;
 	vm_flags_t to_add =3D 0;
=20
@@ -679,7 +681,7 @@ static int apply_mlockall_flags(int flags)
 			to_add |=3D VM_LOCKONFAULT;
 	}
=20
-	for (vma =3D current->mm->mmap; vma ; vma =3D prev->vm_next) {
+	mas_for_each(&mas, vma, ULONG_MAX) {
 		vm_flags_t newflags;
=20
 		newflags =3D vma->vm_flags & VM_LOCKED_CLEAR_MASK;
@@ -687,6 +689,7 @@ static int apply_mlockall_flags(int flags)
=20
 		/* Ignore errors */
 		mlock_fixup(vma, &prev, vma->vm_start, vma->vm_end, newflags);
+		mas_pause(&mas);
 		cond_resched();
 	}
 out:
--=20
2.35.1