Received: by 2002:a05:6a10:a0d1:0:0:0:0 with SMTP id j17csp4425688pxa; Mon, 10 Aug 2020 08:47:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzt40yxwdOMKdlx+ozUeEI5ww6kLAFeMdwUxZVXHVBjntSc2OcYZqI0YCQo1UQAMdpSG+OU X-Received: by 2002:a50:cbc3:: with SMTP id l3mr21741206edi.58.1597074468736; Mon, 10 Aug 2020 08:47:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1597074468; cv=none; d=google.com; s=arc-20160816; b=k2iKh/TUZX1OiwcCXgDjkjR40QlZ+0ACRmcGTeJlz6H1Sbrt79NwCQfO/2qlRgcO5p 21NrfT+GzsPaMYGflDq/d9fKM2mrB+aZC/Fn/FrUPSAZV7YyQOuf0/xSOVBWmpr8CLvu MANZPpDPU+J21uMUJhz8P6rXU0eUKorUCQp34SDBu/PBEHSqXci5NJUTG+p7qHmxyuaG 1DXp0cU9Jn2NZ3DMILRIoOJGXpvulTLr7zOyi37vh900to3pELOm4cuFcwQ+4gmeeyfg UY9oJ0I7vn5qGUhzsW50UhnfAvWVi30+ZJyB7yce8GvlFdYsqB+abSAR7OK6WDOXFkpd dwew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:reply-to:message-id :subject:cc:to:from:date:dkim-signature; bh=LmugU7wnPx2ZTYMNeuoTtyM/9xE1IGuoTy6DIiKZTHg=; b=F8cM5ZTtLbyqCVwi4QU9PLUrD1CvAzO8OgpB7hFBj0HHVtzm0jr06BDSXohAD9Ou8b 0+GB9vvT/oSNKEIebTpEiEbFbux+UEXo0T6j0lxpJ7N2XGPTtnO2ryT7hmPaMnGOU5T6 Kv2JO+NHnMJVKwI7cLvlupJZUC1kQhP8GPtVmILyeAzr0hm/EONmshTQhlwPOFZx3pY/ hVmZRXa/6rktAQT0GaqQ/jjWns6qdAH2RnDiNr3x8V0C/zr+r+eqsXToIO4fYqTbwtX4 yshzXpR4Zy8bPDNt5LJ/n0DnQ8ZsCARiU3rn2qeD6brcKu0PSeELlscc7XfGhD2li/bx QAiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=0X+l8s+t; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id z12si10778437edx.530.2020.08.10.08.47.25; Mon, 10 Aug 2020 08:47:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=0X+l8s+t; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727841AbgHJPqz (ORCPT + 99 others); Mon, 10 Aug 2020 11:46:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:56414 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726406AbgHJPqz (ORCPT ); Mon, 10 Aug 2020 11:46:55 -0400 Received: from paulmck-ThinkPad-P72.home (unknown [50.45.173.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A9E0B2078E; Mon, 10 Aug 2020 15:46:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597074414; bh=j8jGR2gpWSiHYnlpQ+t1j++8nxUJKE0a7/TRK6Ps85U=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=0X+l8s+tl0tCoIQqESJhcYWbW5WNuyg+PexmYrKIBVWBqLtkYyILa2ZQwcRwRL0Vx 2nhRQy8UWf02p9p0IgW4FX76ZjJVqj27oGSNVkkfYEwwfIYEG4bZRXMIR6PyVYIpQP V//VbuJ/POjIz51hrVuZCU2cQc6AckeSy0GRv80I= Received: by paulmck-ThinkPad-P72.home (Postfix, from userid 1000) id 795BA35228C7; Mon, 10 Aug 2020 08:46:54 -0700 (PDT) Date: Mon, 10 Aug 2020 08:46:54 -0700 From: "Paul E. McKenney" To: "Joel Fernandes (Google)" Cc: linux-kernel@vger.kernel.org, Neeraj Upadhyay , Davidlohr Bueso , Jonathan Corbet , Josh Triplett , Lai Jiangshan , linux-doc@vger.kernel.org, Mathieu Desnoyers , Mauro Carvalho Chehab , peterz@infradead.org, Randy Dunlap , rcu@vger.kernel.org, Steven Rostedt , tglx@linutronix.de, vineethrp@gmail.com Subject: Re: [PATCH v4 1/5] rcu/tree: Add a warning if CPU being onlined did not report QS already Message-ID: <20200810154654.GJ4295@paulmck-ThinkPad-P72> Reply-To: paulmck@kernel.org References: <20200807170722.2897328-1-joel@joelfernandes.org> <20200807170722.2897328-2-joel@joelfernandes.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200807170722.2897328-2-joel@joelfernandes.org> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Aug 07, 2020 at 01:07:18PM -0400, Joel Fernandes (Google) wrote: > Currently, rcu_cpu_starting() checks to see if the RCU core expects a > quiescent state from the incoming CPU. However, the current interaction > between RCU quiescent-state reporting and CPU-hotplug operations should > mean that the incoming CPU never needs to report a quiescent state. > First, the outgoing CPU reports a quiescent state if needed. Second, > the race where the CPU is leaving just as RCU is initializing a new > grace period is handled by an explicit check for this condition. Third, > the CPU's leaf rcu_node structure's ->lock serializes these checks. > > This means that if rcu_cpu_starting() ever feels the need to report > a quiescent state, then there is a bug somewhere in the CPU hotplug > code or the RCU grace-period handling code. This commit therefore > adds a WARN_ON_ONCE() to bring that bug to everyone's attention. > > Cc: Paul E. McKenney > Cc: Neeraj Upadhyay > Suggested-by: Paul E. McKenney > Signed-off-by: Joel Fernandes (Google) > --- > kernel/rcu/tree.c | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > index 65e1b5e92319..a49fa3b60faa 100644 > --- a/kernel/rcu/tree.c > +++ b/kernel/rcu/tree.c > @@ -3996,7 +3996,14 @@ void rcu_cpu_starting(unsigned int cpu) > rcu_gpnum_ovf(rnp, rdp); /* Offline-induced counter wrap? */ > rdp->rcu_onl_gp_seq = READ_ONCE(rcu_state.gp_seq); > rdp->rcu_onl_gp_flags = READ_ONCE(rcu_state.gp_flags); > - if (rnp->qsmask & mask) { /* RCU waiting on incoming CPU? */ > + > + /* > + * XXX: The following rcu_report_qs_rnp() is redundant. If the below > + * warning does not fire, consider replacing it with the "else" block, > + * by June 2021 or so (while keeping the warning). Refer to RCU's > + * Requirements documentation for the rationale. Let's suppose that this change is made, and further that in a year or two the "if" statement below is replaced with its "else" block. Now let's suppose that (some years after that) a hard-to-trigger bug makes its way into RCU's CPU-hotplug code that would have resulted in the WARN_ON_ONCE() triggering, but that this bug turns out to be not so hard to trigger in certain large production environments. Let's suppose further that you have moved on to where you are responsible for one of these large production environments. How would this hypothetical RCU/CPU-hotplug bug manifest? Thanx, Paul > + */ > + if (WARN_ON_ONCE(rnp->qsmask & mask)) { /* RCU waiting on incoming CPU? */ > rcu_disable_urgency_upon_qs(rdp); > /* Report QS -after- changing ->qsmaskinitnext! */ > rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags); > -- > 2.28.0.236.gb10cc79966-goog >