Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34592C433F5 for ; Tue, 23 Nov 2021 00:37:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232394AbhKWAky (ORCPT ); Mon, 22 Nov 2021 19:40:54 -0500 Received: from mail.kernel.org ([198.145.29.99]:48384 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231937AbhKWAk1 (ORCPT ); Mon, 22 Nov 2021 19:40:27 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3F70F60F70; Tue, 23 Nov 2021 00:37:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1637627840; bh=Dr4/h+NQHreQm/AZUr+jLC88zQocF18H4nOZHLlG98g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XYuHV6wrp0L6iRBRxd/KwrCljEV4i+AOZ+Gf/s5DIdTPedryxF39SPGVFHIFdoYP+ QUE2+r4jQp+OlGd3KYiISujZc+QwbOWfdovxYPCuTom74qRIblZgexVxw5/k/QdRr2 H3Y2kEG1BWh9qJKwgHArNVA6/M04TiAh9jMzDmqwSojNN2lPIdEbmqTE2oQLR3JVYD APt0AyCmdch5AOfsM03bxAaBcaR0++7L1J7ha3kucxrOiZhGGEHZEcwk8zjFRY+XLu AJueS6663Tj5/yz3qcSn8gGxzSYjkSjKqwx+F1ynwd96ncqu1a6FPnEYTm16nyJFMv CbZxcY67RU+dA== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Josh Triplett , Joel Fernandes , rcu@vger.kernel.org Subject: [PATCH 2/6] rcu/nocb: Prepare nocb_cb_wait() to start with a non-offloaded rdp Date: Tue, 23 Nov 2021 01:37:04 +0100 Message-Id: <20211123003708.468409-3-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211123003708.468409-1-frederic@kernel.org> References: <20211123003708.468409-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In order to be able to toggle the offloaded state from cpusets, a nocb kthread will need to be created for all possible CPUs whenever the "rcu_nocbs=" or "nohz_full=" parameters are passed. Therefore nocb_cb_wait() kthread must prepare to start running on a de-offloaded rdp. Simply move the sleeping condition in the beginning of the kthread callback is enough to prevent from running callbacks before the rdp ever becomes offloaded. Signed-off-by: Frederic Weisbecker Cc: Neeraj Upadhyay Cc: Boqun Feng Cc: Uladzislau Rezki Cc: Josh Triplett Cc: Joel Fernandes --- kernel/rcu/tree_nocb.h | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index cc1165559177..e1cb06840454 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -797,6 +797,18 @@ static void nocb_cb_wait(struct rcu_data *rdp) bool can_sleep = true; struct rcu_node *rnp = rdp->mynode; + do { + swait_event_interruptible_exclusive(rdp->nocb_cb_wq, + nocb_cb_wait_cond(rdp)); + + // VVV Ensure CB invocation follows _sleep test. + if (smp_load_acquire(&rdp->nocb_cb_sleep)) { // ^^^ + WARN_ON(signal_pending(current)); + trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("WokeEmpty")); + } + } while (!nocb_cb_can_run(rdp)); + + local_irq_save(flags); rcu_momentary_dyntick_idle(); local_irq_restore(flags); @@ -849,17 +861,6 @@ static void nocb_cb_wait(struct rcu_data *rdp) if (needwake_state) swake_up_one(&rdp->nocb_state_wq); - - do { - swait_event_interruptible_exclusive(rdp->nocb_cb_wq, - nocb_cb_wait_cond(rdp)); - - // VVV Ensure CB invocation follows _sleep test. - if (smp_load_acquire(&rdp->nocb_cb_sleep)) { // ^^^ - WARN_ON(signal_pending(current)); - trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("WokeEmpty")); - } - } while (!nocb_cb_can_run(rdp)); } /* -- 2.25.1