Date: Wed, 17 Jun 2015 13:37:45 -0700
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Alexei Starovoitov <ast@plumgrid.com>
Cc: Daniel Wagner <daniel.wagner@bmw-carit.de>,
        LKML <linux-kernel@vger.kernel.org>, rostedt@goodmis.org
Subject: Re: call_rcu from trace_preempt
Message-ID: <20150617203745.GR3913@linux.vnet.ibm.com>
Reply-To: paulmck@linux.vnet.ibm.com
References: <20150615230702.GB3913@linux.vnet.ibm.com>
 <557F7764.5060707@plumgrid.com>
 <20150616021458.GE3913@linux.vnet.ibm.com>
 <557FB7E1.6080004@plumgrid.com>
 <20150616122733.GG3913@linux.vnet.ibm.com>
 <558018DD.1080701@monom.org>
 <55805AC5.8020507@plumgrid.com>
 <55812BC1.4010604@bmw-carit.de>
 <5581385D.9060608@bmw-carit.de>
 <5581BEE1.5060302@plumgrid.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <5581BEE1.5060302@plumgrid.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2255
Lines: 46

On Wed, Jun 17, 2015 at 11:39:29AM -0700, Alexei Starovoitov wrote:
> On 6/17/15 2:05 AM, Daniel Wagner wrote:
> >>>Steven's suggestion deferring the work via irq_work results in the same
> >>>stack trace. (Now I get cold feets, without the nice heat from the CPU
> >>>busy looping...)
> >That one still not working. It also makes the system really really slow.
> >I guess I still do something completely wrong.
> 
> tried your irq_work patch. It indeed makes the whole system
> unresponsive. Ctrl-C of hwlathist no longer works and
> it runs out of memory in 20 sec or so of running hwlathist
> on idle system (without parallel hackbench).
> It looks that free_pending flag is racy, so I removed it,
> but it didn't help.
> 
> Also I've tried all sort of other things in rcu including
> add rcu_bpf similar to rcu_sched to make sure that recursive
> call into call_rcu will not be messing rcu_preempt or rcu_sched
> states and instead will be operating on rcu_bpf per-cpu states.
> In theory that should have worked flawlessly and it sort-of did.
> But multiple hackbench runs still managed to crash it.
> So far I think the temp workaround is to stick with array maps
> for probing such low level things like trace_preempt.
> Note that pre-allocation of all elements in hash map also won't
> help, since the problem here is some collision of call_rcu and
> rcu_process_callbacks. I'm pretty sure that kfree_rcu with
> rcu_is_watching patch is ready for this type of abuse.
> The rcu_process_callbacks() path - no yet. I'm still analyzing it.

How about if I just gave you a hook in __call_rcu() itself, just before
it returns, just after the local_irq_restore()?  You could maintain
recursion flags and test the environment, at some point handling any
memory that needed freeing.

The idea would be to use an atomic queue to accumulate the to-be-freed
data, then kfree_rcu() it in the hook if it was safe to do so.

I really don't trust the re-entrancy, especially not in the long term.

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/