Date: Wed, 17 Jun 2015 17:20:43 -0700
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Alexei Starovoitov <ast@plumgrid.com>
Cc: Daniel Wagner <daniel.wagner@bmw-carit.de>,
        LKML <linux-kernel@vger.kernel.org>, rostedt@goodmis.org
Subject: Re: call_rcu from trace_preempt
Message-ID: <20150618002043.GV3913@linux.vnet.ibm.com>
Reply-To: paulmck@linux.vnet.ibm.com
References: <20150616122733.GG3913@linux.vnet.ibm.com>
 <558018DD.1080701@monom.org>
 <55805AC5.8020507@plumgrid.com>
 <55812BC1.4010604@bmw-carit.de>
 <5581385D.9060608@bmw-carit.de>
 <5581BEE1.5060302@plumgrid.com>
 <20150617203745.GR3913@linux.vnet.ibm.com>
 <5581DE3D.1010609@plumgrid.com>
 <20150617213651.GT3913@linux.vnet.ibm.com>
 <558209B8.80405@plumgrid.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <558209B8.80405@plumgrid.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2335
Lines: 50

On Wed, Jun 17, 2015 at 04:58:48PM -0700, Alexei Starovoitov wrote:
> On 6/17/15 2:36 PM, Paul E. McKenney wrote:
> >Well, you do need to have something in each element to allow them to be
> >tracked.  You could indeed use llist_add() to maintain the per-CPU list,
> >and then use llist_del_all() bulk-remove all the elements from the per-CPU
> >list.  You can then pass each element in turn to kfree_rcu().  And yes,
> >I am suggesting that you open-code this, as it is going to be easier to
> >handle your special case then to provide a fully general solution.  For
> >one thing, the general solution would require a full rcu_head to track
> >offset and next.  In contrast, you can special-case the offset.  And
> >ignore the overload special cases.
> 
> yes. all makes sense.
> 
> > Locklessly enqueue onto a per-CPU list, but yes.  The freeing is up to
> 
> yes. per-cpu llist indeed.
> 
> > you -- you get called just before exit from __call_rcu(), and get to
> > figure out what to do.
> >
> > My guess would be if not in interrupt and not recursively invoked,
> > atomically remove all the elements from the list, then pass each to
> > kfree_rcu(), and finally let things take their course from there.
> > The llist APIs look like they would work.
> 
> Above and 'just before the exit from __call_rcu()' part of suggestion
> I still don't understand.
> To avoid reentry into call_rcu I can either create 1 or N new kthreads
> or work_queue and do manual wakeups, but that's very specialized and I
> don't want to permanently waste them, so I'm thinking to llist_add into
> per-cpu llists and do llist_del_all in rcu_process_callbacks() to take
> them from these llists and call kfree_rcu on them.

Another option is to drain the lists the next time you do an allocation.
That would avoid hooking both __call_rcu() and rcu_process_callbacks().

							Thanx, Paul

> The llist_add part will also do:
> if (!rcu_is_watching()) invoke_rcu_core();
> to raise softirq when necessary.
> So at the end it will look like two phase kfree_rcu.
> I'll try to code it up and see it explodes :)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/