Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Message-ID: <1526566209.8616.19.camel@gmx.de>
Subject: Re: cpu stopper threads and load balancing leads to deadlock
From:   Mike Galbraith <efault@gmx.de>
To:     paulmck@linux.vnet.ibm.com
Cc:     Peter Zijlstra <peterz@infradead.org>,
        Matt Fleming <matt@codeblueprint.co.uk>,
        Ingo Molnar <mingo@kernel.org>, linux-kernel@vger.kernel.org,
        Michal Hocko <mhocko@suse.com>
Date:   Thu, 17 May 2018 16:10:09 +0200
In-Reply-To: <20180517140345.GI3803@linux.vnet.ibm.com>
References: <20180503122808.GZ12217@hirez.programming.kicks-ass.net>
         <1525351221.9956.4.camel@gmx.de>
         <20180503124943.GB12217@hirez.programming.kicks-ass.net>
         <1525354359.5576.1.camel@gmx.de>
         <20180503135617.GC12217@hirez.programming.kicks-ass.net>
         <1525357015.5577.2.camel@gmx.de>
         <20180503144450.GD12217@hirez.programming.kicks-ass.net>
         <20180503161231.GI26088@linux.vnet.ibm.com>
         <20180503164508.GG12217@hirez.programming.kicks-ass.net>
         <1526358626.19125.0.camel@gmx.de>
         <20180517140345.GI3803@linux.vnet.ibm.com>
Content-Type: text/plain; charset="ISO-8859-15"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, 2018-05-17 at 07:03 -0700, Paul E. McKenney wrote:
> On Tue, May 15, 2018 at 06:30:26AM +0200, Mike Galbraith wrote:
> 
> > > Something like so perhaps? Mike, can you play around with that? Could
> > > burn your granny and eat your cookies.
> > 
> > Did this get queued anywhere?
> 
> I have not queued it, but given Peter's Signed-off-by and your Tested-by
> I would be happy to do so.

Here's the later. Tested-by: Mike Galbraith <efault@gmx.de>

> > > diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
> > > index 7468de429087..07360523c3ce 100644
> > > --- a/arch/x86/kernel/cpu/mtrr/main.c
> > > +++ b/arch/x86/kernel/cpu/mtrr/main.c
> > > @@ -793,6 +793,9 @@ void mtrr_ap_init(void)
> > >  
> > >  	if (!use_intel() || mtrr_aps_delayed_init)
> > >  		return;
> > > +
> > > +	rcu_cpu_starting(smp_processor_id());
> > > +
> > >  	/*
> > >  	 * Ideally we should hold mtrr_mutex here to avoid mtrr entries
> > >  	 * changed, but this routine will be called in cpu boot time,
> > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > > index 2a734692a581..4dab46950fdb 100644
> > > --- a/kernel/rcu/tree.c
> > > +++ b/kernel/rcu/tree.c
> > > @@ -3775,6 +3775,8 @@ int rcutree_dead_cpu(unsigned int cpu)
> > >  	return 0;
> > >  }
> > >  
> > > +static DEFINE_PER_CPU(int, rcu_cpu_started);
> > > +
> > >  /*
> > >   * Mark the specified CPU as being online so that subsequent grace periods
> > >   * (both expedited and normal) will wait on it.  Note that this means that
> > > @@ -3796,6 +3798,11 @@ void rcu_cpu_starting(unsigned int cpu)
> > >  	struct rcu_node *rnp;
> > >  	struct rcu_state *rsp;
> > >  
> > > +	if (per_cpu(rcu_cpu_started, cpu))
> > > +		return;
> > > +
> > > +	per_cpu(rcu_cpu_started, cpu) = 1;
> > > +
> > >  	for_each_rcu_flavor(rsp) {
> > >  		rdp = per_cpu_ptr(rsp->rda, cpu);
> > >  		rnp = rdp->mynode;
> > > @@ -3852,6 +3859,8 @@ void rcu_report_dead(unsigned int cpu)
> > >  	preempt_enable();
> > >  	for_each_rcu_flavor(rsp)
> > >  		rcu_cleanup_dying_idle_cpu(cpu, rsp);
> > > +
> > > +	per_cpu(rcu_cpu_started, cpu) = 0;
> > >  }
> > >  
> > >  /* Migrate the dead CPU's callbacks to the current CPU. */
> > 
>