Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Liam Howlett <liam.howlett@oracle.com>
To:     "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Hugh Dickins <hughd@google.com>
CC:     Yu Zhao <yuzhao@google.com>
Subject: [PATCH v12 48/69] sched: use maple tree iterator to walk VMAs
Thread-Topic: [PATCH v12 48/69] sched: use maple tree iterator to walk VMAs
Thread-Index: AQHYm97uEDJDG0A1AUmrnukzZag3Vw==
Date:   Wed, 20 Jul 2022 02:17:59 +0000
Message-ID: <20220720021727.17018-49-Liam.Howlett@oracle.com>
References: <20220720021727.17018-1-Liam.Howlett@oracle.com>
In-Reply-To: <20220720021727.17018-1-Liam.Howlett@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?PEeaDMYN+f8nDW7yIoWRbg+FefrJrXFvxWHoPhZcW11yoMlYbXr5wNeUlZ?=
 =?iso-8859-1?Q?pgaS4Zb+v7U8TkWeVQvRuS6xU09VIXh2+BslnJ5Y4XBS6dmqaF1steqSC9?=
 =?iso-8859-1?Q?GP30zCksaISZiAaxwSzsUrl1m5Pq1D/FoTMpSJOot6aA964eeqbzXajNaB?=
 =?iso-8859-1?Q?umDtPQcLA17R06CelEKRt7BBKeCULTkSuCS+3zJjpEI5fngeJ8vDsfJyMi?=
 =?iso-8859-1?Q?eEbQoysLxqDIhLDewJRUg6hckKi+GqSXfWgGHqrGpMU/sIlCjEnZ5Q89vM?=
 =?iso-8859-1?Q?6M8g/tEUCMdME7h0jp9Uu//eUwbq+k6OXfV7tjMP6RsMKfXsFzPYiRv5vm?=
 =?iso-8859-1?Q?Gn32yFpQfh9bL7DLCtTy00yX3menMQIy6mGz5O3gjjQrVPyhQABgFI9CN2?=
 =?iso-8859-1?Q?pOtOqTlmVJtqlwCMwCunwOMjNos7/BtYAlOUE314Gli9UjFRTo0zVW/mmX?=
 =?iso-8859-1?Q?SJ3ceDfvqxgj10kwdG4QBHkOUaCTRrp+zNivxm/Byps03xs5fhqDzgkmvj?=
 =?iso-8859-1?Q?U72XVrxK1beTb7F9w7F8ReZHFkXv9ZVByx2qtdBy0DkGOG2LxiamhJI/jx?=
 =?iso-8859-1?Q?pESTL89EZVtYe6IGi3lQHyixXfe22Scm+KDCzJ9+OLGfiR0se6PvlOY1xX?=
 =?iso-8859-1?Q?b5tzGENd5YFbVtYTg/64xmO/F9ECpte5tDDxeG2gvK2mNtrhl6MhoNZnm2?=
 =?iso-8859-1?Q?kv0YJdrcjRlONrQlvt0YtW6z4+R461L9ABWZ65VesNNuliBPcJiV6nfT/0?=
 =?iso-8859-1?Q?zSuJJnVT5mU32rcqwqzA4bPTEFrJJ67pjcDwIjmXNsB4XMmOACtquGav6I?=
 =?iso-8859-1?Q?DgXY1lUxBREXQ7doWOD1IHG8QKvulj7wPqqcUl+EOj/Dgx4bpsrQ/y6fu/?=
 =?iso-8859-1?Q?+ZEI5Z5iFmlV2w9NoI4wjmZadxaCkF7viz+8fwLdyiAyCMClaTFGlJXxWq?=
 =?iso-8859-1?Q?wwpRmNUvNS8cDLqV8GPeOoHHmcLt/q+ESUd8NFQiTD5R+eKTQ3wi9TxCRF?=
 =?iso-8859-1?Q?xGNu5pzKfhEQjW/yj9xQWlqZld7/F5c6awKhcpdW0m7ADpZ0/ctxyinREy?=
 =?iso-8859-1?Q?oAyDyy+YSXCBYaa+4nCWuYz7GEBp0vCIT/kUg/sMIln4FV6Dbs92UYhXp2?=
 =?iso-8859-1?Q?rgW/P7HDP7em6Mz2V77t3bPkh1UDwf3jgE/Ql98LgjanTgE6x7iKSYKjeY?=
 =?iso-8859-1?Q?L68iHnCMTyZ/xiETX6WvgXgf5Wk99AZaF44Jcg477BM5j2k4k4kUWAsrku?=
 =?iso-8859-1?Q?RHbcsEiC9GKdliDgi/SmsPMQS+JFAltRafmryyYsTsipjmFcXRSZ2zoLXe?=
 =?iso-8859-1?Q?gVql8J98gKeDBqwQb4grvO3sTMr+zz8SafywUeBM9bYqwXbAcGFr2LBb7k?=
 =?iso-8859-1?Q?CspkW8rPF6ZjO3a07xak4iZWwZOYlSUPlWPbB/qUPsR4lf/OobyLuDSqif?=
 =?iso-8859-1?Q?BRXJq0VWc3Y2lwQF3CD9NdCrt7YPed51iEqV4tZPqKmU7qXK/vf8CxeCz3?=
 =?iso-8859-1?Q?SGJvKKdWUsSh6CHFDSzKiaPC6+mx0bK/w/W2YjRya34o+VZ2QRvwVQcu9n?=
 =?iso-8859-1?Q?ko1B6vLxBO04mQ6kfNINGU96Qg+buYh7qy9Trph2buT449WseG4etHf7DV?=
 =?iso-8859-1?Q?unMNufX5tGmqixHrbOQbwa4TY/rU/nOWwDPhhwE3g97+zKShkTSmAYfA?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 980cf325-fa78-4126-72dc-08da69f627fe
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jul 2022 02:17:59.1490
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ohmgHJG+iQ5GzNsh31LsZlPDULiwMcc1c9EgBoNekS8yaJ3rH1a7w8eqS2td+MZqu9IqeoL6rY0dva5ThLjKfg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR10MB3797
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1
 definitions=2022-07-19_10,2022-07-19_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0
 mlxlogscore=999 malwarescore=0 adultscore=0 mlxscore=0 spamscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2206140000 definitions=main-2207200008
X-Proofpoint-GUID: FRUG40C6IpOY1bkOXURtAUiKGMkbahp8
X-Proofpoint-ORIG-GUID: FRUG40C6IpOY1bkOXURtAUiKGMkbahp8
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: "Matthew Wilcox (Oracle)" <willy@infradead.org>

The linked list is slower than walking the VMAs using the maple tree.  We
can't use the VMA iterator here because it doesn't support moving to an
earlier position.

Link: https://lkml.kernel.org/r/20220504011345.662299-33-Liam.Howlett@oracl=
e.com
Link: https://lkml.kernel.org/r/20220621204632.3370049-49-Liam.Howlett@orac=
le.com
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Liam R. Howlett <Liam.Howlett@Oracle.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Howells <dhowells@redhat.com>
Cc: SeongJae Park <sj@kernel.org>
Cc: Will Deacon <will@kernel.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
 kernel/sched/fair.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 77b2048a9326..e8202b5cd3d5 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -2734,6 +2734,7 @@ static void task_numa_work(struct callback_head *work=
)
 	struct task_struct *p =3D current;
 	struct mm_struct *mm =3D p->mm;
 	u64 runtime =3D p->se.sum_exec_runtime;
+	MA_STATE(mas, &mm->mm_mt, 0, 0);
 	struct vm_area_struct *vma;
 	unsigned long start, end;
 	unsigned long nr_pte_updates =3D 0;
@@ -2790,13 +2791,16 @@ static void task_numa_work(struct callback_head *wo=
rk)
=20
 	if (!mmap_read_trylock(mm))
 		return;
-	vma =3D find_vma(mm, start);
+	mas_set(&mas, start);
+	vma =3D mas_find(&mas, ULONG_MAX);
 	if (!vma) {
 		reset_ptenuma_scan(p);
 		start =3D 0;
-		vma =3D mm->mmap;
+		mas_set(&mas, start);
+		vma =3D mas_find(&mas, ULONG_MAX);
 	}
-	for (; vma; vma =3D vma->vm_next) {
+
+	for (; vma; vma =3D mas_find(&mas, ULONG_MAX)) {
 		if (!vma_migratable(vma) || !vma_policy_mof(vma) ||
 			is_vm_hugetlb_page(vma) || (vma->vm_flags & VM_MIXEDMAP)) {
 			continue;
--=20
2.35.1