Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Liam Howlett <liam.howlett@oracle.com>
To:     "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Yu Zhao <yuzhao@google.com>, Hugh Dickins <hughd@google.com>
Subject: [PATCH v11 48/69] sched: use maple tree iterator to walk VMAs
Thread-Topic: [PATCH v11 48/69] sched: use maple tree iterator to walk VMAs
Thread-Index: AQHYmYd4R1T1u0ViKUGx93yOhL10aA==
Date:   Sun, 17 Jul 2022 02:46:53 +0000
Message-ID: <20220717024615.2106835-49-Liam.Howlett@oracle.com>
References: <20220717024615.2106835-1-Liam.Howlett@oracle.com>
In-Reply-To: <20220717024615.2106835-1-Liam.Howlett@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?5kN/dzEQ8Pr0qYs12ZFZ1GN4Z2KKLtF2+CTmpfuQVeoldR0Znb0WlUMWha?=
 =?iso-8859-1?Q?xABRGC4uhQtCHKmU9xiV5wWNPnsRBv/ByrxFGfOdQnykNmk+/gdVWVxvJT?=
 =?iso-8859-1?Q?kbH0pzm8vHg/K+W59z9rJ76k7EKmGEeXcHH8rT8PZMO0jGsR9LRAZZDxeE?=
 =?iso-8859-1?Q?buB7Qb9xOwNH2ajVdp9nCa+8IeY+o+/hcLtc3zNwBj03MakDJ9mlzPCQgs?=
 =?iso-8859-1?Q?fiSjayWMzn6YzEbDlKIoyOewVAtGUSO7Sc2RvJICq+8OU2Qf4FMd31zRXx?=
 =?iso-8859-1?Q?wAZ/T6lIh1kz3y6K+ddJ1jk5LlQ75HOOK6kERtXfevTBlRXGfhsSwWFCjx?=
 =?iso-8859-1?Q?H7uybqrclpJ49U706/H7rvjXrObTbxCUIfBzHQWAAO53OmyFoe2maTjL4+?=
 =?iso-8859-1?Q?DhCiNv2YS6T5ODNycIXByp6Ln1JsygpsKp7Dj+EjEjgHFFIz/5MaCncsaI?=
 =?iso-8859-1?Q?YHwYCMa8MG5jy4CWv8gZWYlmLGKRNa+dmsQhT2Vr45PxSao+J16l6IAWc2?=
 =?iso-8859-1?Q?pANTfpFwF1oXlX4Geaz+yzxH4hWFPV4Zz+k2a3LMm30qwhLIRygEDkgO2Y?=
 =?iso-8859-1?Q?mokPOcMOgBK49V/16JKX3ZxVJHNn/BWW0C1e9EuRjfAy49wMekIPuHAzyA?=
 =?iso-8859-1?Q?P2KgJVpfF7JKkh9btcaP0mOxvTPXBF30RvRYg8FbJAEOK8Brwk+SR5Y6Rf?=
 =?iso-8859-1?Q?zojFy/RWkP2pSXX2adWabRUcjP/9bLcoiwswBbQ9g7PH0Dv4zh14syzUT6?=
 =?iso-8859-1?Q?ImAZ8ng17XLbuPIIsd+CXrmvTjlP+vL/0n6xZij3FvTamACTRAlgBALKer?=
 =?iso-8859-1?Q?A0Lvz1taNcHXQiJ7ISW3VhgUWE6iZD519NW79pBNhMOi6Y/6nwsURnPxAS?=
 =?iso-8859-1?Q?qzEuUO7JKwo+skI0MGUqVc/FznTGSORK1IImJvYUwOC8puP8h6Hm8dVaHL?=
 =?iso-8859-1?Q?1QAq6puFOh7JFSbBkS+MqMjoIfHvO9ZlAkFhJfwrLc4dRPccccaUIzG8tu?=
 =?iso-8859-1?Q?g5ISjzPvn8GwWcf2RPjmQaKbcc7QTgqFQFBgYbadt9rJoEPQvz9CpUGHJp?=
 =?iso-8859-1?Q?YAYDjXqimBRlvk0yhfj0KbF2fC6yjYi+J2ACTUy4hFeAQljlAz/HD8Qa6u?=
 =?iso-8859-1?Q?bANj1yivPX6S6wGWUZIxOOQE5QA88geJfqdDvge/yxMGvX1LXPIELupZa7?=
 =?iso-8859-1?Q?7EA9l02Dp8B5oH4RtKmHDVjRNPCkjXxoXpEdmdTbE/TlxheyJJAkBMcHvq?=
 =?iso-8859-1?Q?epSZwGdxmAnlSAcVKVCoE3BF3xLBQd1Kou/MVp0wUIIl19n6iRR3scNc8w?=
 =?iso-8859-1?Q?2WCQOtnyGY7BPu8i3vwx9+cjZeXUTvYteSxUS2ldOJbR48e5hGtZ7auQzB?=
 =?iso-8859-1?Q?YwgTteqZAJfjtC+4vGfd2YnAMy7pBj5rdXXDIgo90YiWIY9MbcQN21Vmyq?=
 =?iso-8859-1?Q?ljBRxsBqJ5jGf/LvF2jEaO4xCZWiMTJH0+86YE8tJoM9Kcj7V+9y3qX+2w?=
 =?iso-8859-1?Q?gzYyOKen3gXJjJOAoCmd4VvXR/JGBabbbm1Z8HB41Wv0u6Sn2iSMOjSj1A?=
 =?iso-8859-1?Q?a4g/YfE9sX0RG9bfp1FDFSxYyuO8L+z6AnoRxZ+LJktwa561W9z2qE5ksI?=
 =?iso-8859-1?Q?gifECGyRrRR3rNWvCAA+smhghhEIk9nCgub5s+5l0i3X4ZJed+4AHL6g?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 73d3fb40-a6ce-47f7-b384-08da679eb0e6
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jul 2022 02:46:53.1305
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: CQ8hy8e0Nn80YD9d0gWvJD45hSjuzUzjDY/APwvE4dtqKluwfFC0as/DtmVS7fc5zfsJHPNJsyd49mB7lwCP2g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR10MB2799
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1
 definitions=2022-07-17_01,2022-07-15_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 mlxlogscore=999
 suspectscore=0 phishscore=0 adultscore=0 spamscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2206140000
 definitions=main-2207170010
X-Proofpoint-GUID: mWCeVw7hp-tRN0EPDUbG-EFdDqK8lcVj
X-Proofpoint-ORIG-GUID: mWCeVw7hp-tRN0EPDUbG-EFdDqK8lcVj
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: "Matthew Wilcox (Oracle)" <willy@infradead.org>

The linked list is slower than walking the VMAs using the maple tree.  We
can't use the VMA iterator here because it doesn't support moving to an
earlier position.

Link: https://lkml.kernel.org/r/20220504011345.662299-33-Liam.Howlett@oracl=
e.com
Link: https://lkml.kernel.org/r/20220621204632.3370049-49-Liam.Howlett@orac=
le.com
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Liam R. Howlett <Liam.Howlett@Oracle.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Howells <dhowells@redhat.com>
Cc: SeongJae Park <sj@kernel.org>
Cc: Will Deacon <will@kernel.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
 kernel/sched/fair.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 77b2048a9326..e8202b5cd3d5 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -2734,6 +2734,7 @@ static void task_numa_work(struct callback_head *work=
)
 	struct task_struct *p =3D current;
 	struct mm_struct *mm =3D p->mm;
 	u64 runtime =3D p->se.sum_exec_runtime;
+	MA_STATE(mas, &mm->mm_mt, 0, 0);
 	struct vm_area_struct *vma;
 	unsigned long start, end;
 	unsigned long nr_pte_updates =3D 0;
@@ -2790,13 +2791,16 @@ static void task_numa_work(struct callback_head *wo=
rk)
=20
 	if (!mmap_read_trylock(mm))
 		return;
-	vma =3D find_vma(mm, start);
+	mas_set(&mas, start);
+	vma =3D mas_find(&mas, ULONG_MAX);
 	if (!vma) {
 		reset_ptenuma_scan(p);
 		start =3D 0;
-		vma =3D mm->mmap;
+		mas_set(&mas, start);
+		vma =3D mas_find(&mas, ULONG_MAX);
 	}
-	for (; vma; vma =3D vma->vm_next) {
+
+	for (; vma; vma =3D mas_find(&mas, ULONG_MAX)) {
 		if (!vma_migratable(vma) || !vma_policy_mof(vma) ||
 			is_vm_hugetlb_page(vma) || (vma->vm_flags & VM_MIXEDMAP)) {
 			continue;
--=20
2.35.1