Received: by 2002:a05:7412:8d10:b0:f3:1519:9f41 with SMTP id bj16csp4665563rdb; Tue, 12 Dec 2023 06:14:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IFEgRXr7XGxNSvp3JA8R1OQTVxNqBkjDhYrHuerdkHqbOPeHXwRxvY7woyRFx8YhvYjnmH3 X-Received: by 2002:a05:6a00:8b03:b0:6ce:2731:a08b with SMTP id if3-20020a056a008b0300b006ce2731a08bmr5211870pfb.58.1702390467980; Tue, 12 Dec 2023 06:14:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702390467; cv=none; d=google.com; s=arc-20160816; b=nkEpR0Ji5XwMA9AGuKcGzoIHYQs5A3cO+Eo2v/VbHbqFbfSp9i+f5gm2K3uPqibMMR yfDIStZSzLpvR/x4NGosVMX4gIbOyl5/iUkcBQxfSPQJ+cw59sBLCNsqDpYSYUM/NJ41 CnOSzvkV8KpNvlWA6RGY19hcKZlxKtnX4x6WJrgKdnUkFlxU2Q0FYymL1CgQwpHE3Chg /GZ5kqV+T3kmdi5vfKLh2gj6o0mTXVWG9lajL9Krm+9F5akAfAKPy0oV79w1bqv7wcHR oSRMVYMg6XvlYcgzlUVVE5m8zyGf7VK85c+DCjAMRiEzr8nKO2elytV+hjgHtVrOXNdp 4zoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:from :references:cc:to:content-language:subject:user-agent:mime-version :date:message-id:dkim-signature; bh=CAyXmKG4VmN5g0uVlHvg+/MKGQjcw6UMGWYhNPch54E=; fh=UMyPF/gBDSCiv7KMsdzg0C/HbLfP3jyBN1B9MgsIMe8=; b=XIalJf8x8iPMci5jvCjyORm84+2dO5wnzwHi0ETk6FeemTJcftIMz6ch1nH7yNMd5Y KglYkeZMduRauWYm0MTyeemdCODrs0vDAzBquaPRIC/BG3zRNo803fFoqF9UHG5SPIzG LIYGX+L+Gn0KpMrRcpf9dwbF6Akwoc6qXHLuPMUtj/rmY7lQ2APW05Ht7HDV/0fiVA7E BTLh0GQBMU6cX3eG/HrsUAxjLt8fzrfKGCvBgAKTql8PkiezgAFDmytw22W++VJYfgSz Az+x7mGdly1SaTxyQyc1fRAELga67M3mPsNk9FCbbzAxt66hcZ0wQoTo9LQgwHcio9uD GUYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@efficios.com header.s=smtpout1 header.b=x9AEh33a; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=efficios.com Return-Path: Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id u17-20020a656711000000b005bd2b888545si7598803pgf.119.2023.12.12.06.14.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 06:14:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@efficios.com header.s=smtpout1 header.b=x9AEh33a; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=efficios.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 960BB804E6B5; Tue, 12 Dec 2023 06:14:26 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376448AbjLLOOR (ORCPT + 99 others); Tue, 12 Dec 2023 09:14:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232555AbjLLOOQ (ORCPT ); Tue, 12 Dec 2023 09:14:16 -0500 Received: from smtpout.efficios.com (unknown [IPv6:2607:5300:203:b2ee::31e5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 586DE8F for ; Tue, 12 Dec 2023 06:14:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1702390460; bh=CZlz57rWloWiitqrkGaj9Jzas8iR7il2LGCiEK9NI+8=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=x9AEh33aS1IU8cNjHWimEjSV5FcdC96JJV/XVb5wmeu3LQLZP2PwLg4DVxgMQQUlY 7ATz53jDx/pinW8gD/upRkKXS6ADlepm7Hil/wUkw3wk2+aaGZPE5DVeI7yn+t46pd qDHwujasnlHU0ZZj5KN3v+kJMLxV3Pf0YigY2zPnqCEcMwNvTApRhZ39M4M0vVxDPF KiY0lmcNc6WioBwjzqoo0vb2QCR22HiNkgrhuqC32gmOeYsqPaQoAj/Dcgx8NmIeg0 l94z3gkfOkt+cUgqO3N2j25W8eU7x328+xppya1yY37kx9UrAY+6MV5wjGVAniSzAu VMiJ9hcOfWusQ== Received: from [172.16.0.134] (192-222-143-198.qc.cable.ebox.net [192.222.143.198]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4SqLG36XGTzGKC; Tue, 12 Dec 2023 09:14:19 -0500 (EST) Message-ID: Date: Tue, 12 Dec 2023 09:14:19 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 05/35] sched: add cpumask_find_and_set() and use it in __mm_cid_get() Content-Language: en-US To: Yury Norov , linux-kernel@vger.kernel.org, Andy Shevchenko , Rasmus Villemoes , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider Cc: Jan Kara , Mirsad Todorovac , Matthew Wilcox , Maxim Kuvyrkov , Alexey Klimov , Bart Van Assche , Sergey Shtylyov References: <20231212022749.625238-1-yury.norov@gmail.com> <20231212022749.625238-6-yury.norov@gmail.com> From: Mathieu Desnoyers In-Reply-To: <20231212022749.625238-6-yury.norov@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED,RDNS_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 12 Dec 2023 06:14:26 -0800 (PST) On 2023-12-11 21:27, Yury Norov wrote: > __mm_cid_get() uses __mm_cid_try_get() helper to atomically acquire a > bit in mm cid mask. Now that we have atomic find_and_set_bit(), we can > easily extend it to cpumasks and use in the scheduler code. > > cpumask_find_and_set() considers cid mask as a volatile region of memory, > as it actually is in this case. So, if it's changed while search is in > progress, KCSAN wouldn't fire warning on it. > > CC: Mathieu Desnoyers > CC: Peter Zijlstra > Signed-off-by: Yury Norov Reviewed-by: Mathieu Desnoyers > --- > include/linux/cpumask.h | 12 ++++++++++++ > kernel/sched/sched.h | 14 +++++--------- > 2 files changed, 17 insertions(+), 9 deletions(-) > > diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h > index cfb545841a2c..c2acced8be4e 100644 > --- a/include/linux/cpumask.h > +++ b/include/linux/cpumask.h > @@ -271,6 +271,18 @@ unsigned int cpumask_next_and(int n, const struct cpumask *src1p, > small_cpumask_bits, n + 1); > } > > +/** > + * cpumask_find_and_set - find the first unset cpu in a cpumask and > + * set it atomically > + * @srcp: the cpumask pointer > + * > + * Return: >= nr_cpu_ids if nothing is found. > + */ > +static inline unsigned int cpumask_find_and_set(volatile struct cpumask *srcp) > +{ > + return find_and_set_bit(cpumask_bits(srcp), small_cpumask_bits); > +} > + > /** > * for_each_cpu - iterate over every cpu in a mask > * @cpu: the (optionally unsigned) integer iterator > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > index 2e5a95486a42..2ce9112de89b 100644 > --- a/kernel/sched/sched.h > +++ b/kernel/sched/sched.h > @@ -3347,23 +3347,19 @@ static inline void mm_cid_put(struct mm_struct *mm) > > static inline int __mm_cid_try_get(struct mm_struct *mm) > { > - struct cpumask *cpumask; > - int cid; > + struct cpumask *cpumask = mm_cidmask(mm); > + int cid = nr_cpu_ids; > > - cpumask = mm_cidmask(mm); > /* > * Retry finding first zero bit if the mask is temporarily > * filled. This only happens during concurrent remote-clear > * which owns a cid without holding a rq lock. > */ > - for (;;) { > - cid = cpumask_first_zero(cpumask); > - if (cid < nr_cpu_ids) > - break; > + while (cid >= nr_cpu_ids) { > + cid = cpumask_find_and_set(cpumask); > cpu_relax(); > } > - if (cpumask_test_and_set_cpu(cid, cpumask)) > - return -1; > + > return cid; > } > -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com