Received: by 2002:ac0:bc90:0:0:0:0:0 with SMTP id a16csp628418img; Fri, 22 Mar 2019 05:28:38 -0700 (PDT) X-Google-Smtp-Source: APXvYqwooTp48Tg2Vjv+LxlByW0V7BwZgoL9SgdxSz7llni1oHdYn0Sgki3eNa2e669aw8b/htMG X-Received: by 2002:a63:5c66:: with SMTP id n38mr8632131pgm.15.1553257717996; Fri, 22 Mar 2019 05:28:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553257717; cv=none; d=google.com; s=arc-20160816; b=vHKoHtAUiov3mZXAY8VX+TkBVEAIBIvYSyGiIP1Btnk1txesOIxJmBi59qVhMDt2+k sjfIXtz5jsIg6tVmu50S4PTV+ZqEVxF6Ft4IikiH4uNKJgI6saOm7PJhE5m592tLnDC/ qTnj78zUYJYH2xdWSCScBUQQtuF7TsdYlWnd0Ovf3Y9SDXzWcy6ZjhuQ+9Q9ACWTH0p6 QUOUy5aXQIbHHypyagJjqyfxI6U4CEyJOS9eo7zInZb1l95fAOKCLb4v66fGRbvth2XJ zjW9KLg82ge+XytNY0WE9wMYO9VzATf9kFkwfXNjeIN04Kl0MO9ylEoiIp6EUI+Ljzn8 KomA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=erQgLZhu+4L8oyICR98O4nxM9EnLkBpPw1PAQAOIkrw=; b=UKZ5ylT0Lr/1R7HLtV47H0hAt9cDMOmTJ1GvmdCcg4X39c/HV8ZLhrc0HjMd6ssEkN Cnw5zIT7v58a+NpyWfgMOI8WRoR3Bgb9B818HZXsUZtyQciWMSUxLbzBNQukRrO5HB8+ laMNNPLP++t6zXaGXGZu5TbuIwVdFZyV5Bv+VfTaAqw/IUzbdRQfqkCiHejdIvZ9+SoP rVS8Ko2sC7XKLdY4RyJYbZSTDGdEJYMiAT+rmctUayTPziB/QN/rq+sz4xxyBjzIIV8j 2Z1iLPs3viUJKFWwQGulr/THypuQrZAViuUMBGp7gVJ0n06cusUF0k4mm0KlnCdph3lE x7bg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=2fMCJDUn; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l6si6470576pgq.305.2019.03.22.05.28.22; Fri, 22 Mar 2019 05:28:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=2fMCJDUn; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390997AbfCVM1V (ORCPT + 99 others); Fri, 22 Mar 2019 08:27:21 -0400 Received: from mail.kernel.org ([198.145.29.99]:38846 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731406AbfCVM1S (ORCPT ); Fri, 22 Mar 2019 08:27:18 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2A1602070D; Fri, 22 Mar 2019 12:27:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1553257637; bh=ozLqMbItR6ADgErcU+L5iEwMHs0FzBf9wbCS16Vn5B4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=2fMCJDUny3sJbi4c8flCbCZfPs/VK1pg1FHbe33mXKIe6uVwVt6gLLfF305ZAqn5R GGN5+7lVYGwhvs9PVgwSjVOH7EMvsJxxwT4MbJGTddyqPKMA/gixicEBd/mI17Tat1 2XsgSfhCUEGGybSIFOxJ+HiYPLm/FTvAujoe0Ytg= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "He, Bo" , "Zhang, Jun" , "Paul E. McKenney" , "xiao, jin" , Jie A Subject: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt Date: Fri, 22 Mar 2019 12:15:45 +0100 Message-Id: <20190322111220.102008032@linuxfoundation.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190322111210.465931067@linuxfoundation.org> References: <20190322111210.465931067@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review X-Patchwork-Hint: ignore MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 3.18-stable review patch. If anyone has any objections, please let me know. ------------------ From: Zhang, Jun commit 1d1f898df6586c5ea9aeaf349f13089c6fa37903 upstream. The rcu_gp_kthread_wake() function is invoked when it might be necessary to wake the RCU grace-period kthread. Because self-wakeups are normally a useless waste of CPU cycles, if rcu_gp_kthread_wake() is invoked from this kthread, it naturally refuses to do the wakeup. Unfortunately, natural though it might be, this heuristic fails when rcu_gp_kthread_wake() is invoked from an interrupt or softirq handler that interrupted the grace-period kthread just after the final check of the wait-event condition but just before the schedule() call. In this case, a wakeup is required, even though the call to rcu_gp_kthread_wake() is within the RCU grace-period kthread's context. Failing to provide this wakeup can result in grace periods failing to start, which in turn results in out-of-memory conditions. This race window is quite narrow, but it actually did happen during real testing. It would of course need to be fixed even if it was strictly theoretical in nature. This patch does not Cc stable because it does not apply cleanly to earlier kernel versions. Fixes: 48a7639ce80c ("rcu: Make callers awaken grace-period kthread") Reported-by: "He, Bo" Co-developed-by: "Zhang, Jun" Co-developed-by: "He, Bo" Co-developed-by: "xiao, jin" Co-developed-by: Bai, Jie A Signed-off: "Zhang, Jun" Signed-off: "He, Bo" Signed-off: "xiao, jin" Signed-off: Bai, Jie A Signed-off-by: "Zhang, Jun" [ paulmck: Switch from !in_softirq() to "!in_interrupt() && !in_serving_softirq() to avoid redundant wakeups and to also handle the interrupt-handler scenario as well as the softirq-handler scenario that actually occurred in testing. ] Signed-off-by: Paul E. McKenney Link: https://lkml.kernel.org/r/CD6925E8781EFD4D8E11882D20FC406D52A11F61@SHSMSX104.ccr.corp.intel.com Signed-off-by: Greg Kroah-Hartman --- kernel/rcu/tree.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1399,15 +1399,23 @@ static int rcu_future_gp_cleanup(struct } /* - * Awaken the grace-period kthread for the specified flavor of RCU. - * Don't do a self-awaken, and don't bother awakening when there is - * nothing for the grace-period kthread to do (as in several CPUs - * raced to awaken, and we lost), and finally don't try to awaken - * a kthread that has not yet been created. + * Awaken the grace-period kthread. Don't do a self-awaken (unless in + * an interrupt or softirq handler), and don't bother awakening when there + * is nothing for the grace-period kthread to do (as in several CPUs raced + * to awaken, and we lost), and finally don't try to awaken a kthread that + * has not yet been created. If all those checks are passed, track some + * debug information and awaken. + * + * So why do the self-wakeup when in an interrupt or softirq handler + * in the grace-period kthread's context? Because the kthread might have + * been interrupted just as it was going to sleep, and just after the final + * pre-sleep check of the awaken condition. In this case, a wakeup really + * is required, and is therefore supplied. */ static void rcu_gp_kthread_wake(struct rcu_state *rsp) { - if (current == rsp->gp_kthread || + if ((current == rsp->gp_kthread && + !in_interrupt() && !in_serving_softirq()) || !ACCESS_ONCE(rsp->gp_flags) || !rsp->gp_kthread) return;