Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Liam Howlett <liam.howlett@oracle.com>
To:     "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>
Subject: [PATCH v5 49/70] sched: Use maple tree iterator to walk VMAs
Thread-Topic: [PATCH v5 49/70] sched: Use maple tree iterator to walk VMAs
Thread-Index: AQHYF95+zNXtgQNtoEqQVrQg1QPuhg==
Date:   Wed, 2 Feb 2022 02:42:18 +0000
Message-ID: <20220202024137.2516438-50-Liam.Howlett@oracle.com>
References: <20220202024137.2516438-1-Liam.Howlett@oracle.com>
In-Reply-To: <20220202024137.2516438-1-Liam.Howlett@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?8GZsycb648uhghAJBKZuP9ndxYkzSBUeDa+6EHcDWe0Y33eIcrc+/ynFNn?=
 =?iso-8859-1?Q?Puzhm5RagInT7nUz0bizvBKQOFnr/n1U4p9285y9AATJgWgj5BoiGZmRvC?=
 =?iso-8859-1?Q?N7Db4kgkS944MD7/LwngFLvk76oN2dOHWFOhzOU/I/aUrda77H4VQxF8Yf?=
 =?iso-8859-1?Q?avyGGK4b3mvDqPBR/UOy0vA/7HMhH5uA9SqTvNJ6kj0aBwypHF0YsNKXvh?=
 =?iso-8859-1?Q?x8D1oxx3QCASIBQ+7ODEhlI3xIUNZlkKQOdLrDuJu4UlLPloufikvfOMAs?=
 =?iso-8859-1?Q?/7gIEOM5Z/vuosXBuDovXuW+oQX62bubwDZbhuzIxymlIzNwj1MDNxSFqt?=
 =?iso-8859-1?Q?DHzkr36YaeNAp4C//S1wYqQXI/PQAfA0/EDvaksoVe8fjR2IMvkwB0TQw+?=
 =?iso-8859-1?Q?awww3J+1SpIifiNDDburDrNniRmbKwGWGBmJ0WLM/pQ4Ibmwbp53SSPZKG?=
 =?iso-8859-1?Q?+yobx7mQoXuN1exINORAscE+yxYwAdn+XdovvuDk4AHTMiOBmiHJAgBeO6?=
 =?iso-8859-1?Q?jaaneZ4c+7ANVYZL/B/rctrOgfBDDGBR0mH+lp6jFIQFYMmblUrizpe6i1?=
 =?iso-8859-1?Q?k4dqEi89VBfqowfnCDVkw7SYQyiLUtrGT7xGOCN9goqxUraHyaKii3L9tZ?=
 =?iso-8859-1?Q?PDFe2jM2EBBdu6U9JDw7znWUDRVqcZNaCrQg/Io0lziUvkcQchkPDS2Q1u?=
 =?iso-8859-1?Q?2THPkJDTqaLUBmxBJDyabzSGcC54qmbpXWw9I+9DvsJxxkhHQt8az6pcpp?=
 =?iso-8859-1?Q?l/y+2WDLgCtvilVBYXs+z/r8abo0yKm8jqNiNguJgYrSg39SDzAwXNE1Ko?=
 =?iso-8859-1?Q?VOEZ/HWiw5ry4h6kGwTLOUWCJd6JYvzBsT0itjJWAZhrBuLlE1y9O6MH/P?=
 =?iso-8859-1?Q?1U6+Xraz3c0AOkqSjd+UGM2gJnUYWZQ2Box2klWxMKHJw7PDO7QVbjDx9f?=
 =?iso-8859-1?Q?s1E1JSSmbNs7EBdMXuT0I5wJ7alYWSGEUvfqqYN1OSZ86G86f1QxXRKCiK?=
 =?iso-8859-1?Q?qK+popUtMRmTMh4sZgvhDHof3mOM3WNxHslwraZtV7wA8lxOFinyKJrz+M?=
 =?iso-8859-1?Q?cE89pdX9fwg54pDt3Nfgc26R+IBgCv+AoW5jeTkXz3f9C6cxrRQkkBJc2d?=
 =?iso-8859-1?Q?bCqGrKbCGBaw8TU08o4RxO0pT6vrLXptw83N3NRl8Dj+Zmc6GirYjCCTcJ?=
 =?iso-8859-1?Q?zuoQsOVP39Med52i3EvIUzyjd8FKVZnqdQoneZNmVUhkQe+69FZ5JcLB0P?=
 =?iso-8859-1?Q?crnsK9uikhIuc56HnZmGcC/CSBjp6Y1tXRQkHUimg8pSIatjm+xXf1+kfd?=
 =?iso-8859-1?Q?A8lL+RUcn65ZkVG6kGDEraFozGpdiN3xriXCx8x3Vxz91AfjIMGUfYC6hT?=
 =?iso-8859-1?Q?lTP2U5XbalYcDBCl0RiObIWzh9GyrRlGJ4d/ncuXv0Pw17t3LQXYS8z5OY?=
 =?iso-8859-1?Q?bGbaU/VpR/GcJEms3Me1R2QlWOlmlX3ThDDjoVaUXzD5msHcUphQKzJK0H?=
 =?iso-8859-1?Q?8wL7GmG72xFiAMrJ4zWGiGGIvWU7uODjwQIBrTIUJYZ00aEv99iKAlvc7f?=
 =?iso-8859-1?Q?aYEcP+JQsL+If6sSP+rLt+YsHm/447dFXvGdOXCqUg04uJS8MYQ7E6bkZP?=
 =?iso-8859-1?Q?VlHk2rKSCj8F66oYo2zJLlfMYqMtbape26j7VRjf8Hu28uSStgKAcRvEhz?=
 =?iso-8859-1?Q?3jsDcPgqu3On9mh7uu8=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 743990d1-4183-4a22-e810-08d9e5f5b6c2
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Feb 2022 02:42:18.6728
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: lylDLzBzR6Td40jTjx/fM50AiaY4OmgWpg1t5yisyFN0h2BHOAow78CHy+qhkVTq532pgWyvoJOGiRUMGpTkaQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB5865
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10245 signatures=673430
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0
 mlxscore=0 adultscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000
 definitions=main-2202020010
X-Proofpoint-ORIG-GUID: ZKPcFaTnae90XuQhUny10K0I5hMNPL0L
X-Proofpoint-GUID: ZKPcFaTnae90XuQhUny10K0I5hMNPL0L
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: "Liam R. Howlett" <Liam.Howlett@Oracle.com>

The linked list is slower than walking the VMAs using the maple tree.
We can't use the VMA iterator here because it doesn't support
moving to an earlier position.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Liam R. Howlett <Liam.Howlett@Oracle.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 kernel/sched/fair.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 5146163bfabb..f0ac5dfe2ab7 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -2671,6 +2671,7 @@ static void task_numa_work(struct callback_head *work=
)
 	struct task_struct *p =3D current;
 	struct mm_struct *mm =3D p->mm;
 	u64 runtime =3D p->se.sum_exec_runtime;
+	MA_STATE(mas, &mm->mm_mt, 0, 0);
 	struct vm_area_struct *vma;
 	unsigned long start, end;
 	unsigned long nr_pte_updates =3D 0;
@@ -2727,13 +2728,16 @@ static void task_numa_work(struct callback_head *wo=
rk)
=20
 	if (!mmap_read_trylock(mm))
 		return;
-	vma =3D find_vma(mm, start);
+	mas_set(&mas, start);
+	vma =3D mas_find(&mas, ULONG_MAX);
 	if (!vma) {
 		reset_ptenuma_scan(p);
 		start =3D 0;
-		vma =3D mm->mmap;
+		mas_set(&mas, start);
+		vma =3D mas_find(&mas, ULONG_MAX);
 	}
-	for (; vma; vma =3D vma->vm_next) {
+
+	for (; vma; vma =3D mas_find(&mas, ULONG_MAX)) {
 		if (!vma_migratable(vma) || !vma_policy_mof(vma) ||
 			is_vm_hugetlb_page(vma) || (vma->vm_flags & VM_MIXEDMAP)) {
 			continue;
--=20
2.34.1