Received-SPF: pass (google.com: domain of linux-kernel+bounces-213514-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Thu, 13 Jun 2024 08:15:06 -0700
From: "Paul E. McKenney" <paulmck@kernel.org>
To: Keith Busch <kbusch@kernel.org>
Cc: Nilay Shroff <nilay@linux.ibm.com>, l@kbusch-mbp.dhcp.thefacebook.com,
	Keith Busch <kbusch@meta.com>, linux-nvme@lists.infradead.org,
	linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	hch@lst.de, sagi@grimberg.me, davidgow@google.com,
	akpm@linux-foundation.org, venkat88@linux.vnet.ibm.com
Subject: Re: [PATCH 1/2] list: introduce a new cutting helper
Message-ID: <73b994eb-c689-48e0-b09c-a414041a0525@paulmck-laptop>
Reply-To: paulmck@kernel.org
References: <20240612155135.3060667-1-kbusch@meta.com>
 <f0e4c51c-8227-4f5c-876f-38fbb4a0e1bf@linux.ibm.com>
 <ZmrscxG51gFRDVlM@kbusch-mbp>
 <c475f0d8-3bc9-4d65-8fce-586f4b75b4fc@linux.ibm.com>
 <ZmsD_HDLBQAqOOU3@kbusch-mbp.dhcp.thefacebook.com>
 <31eb40f9-d68d-4348-b1fd-3cf057939297@paulmck-laptop>
 <ZmsGfle1aZQauzRb@kbusch-mbp.dhcp.thefacebook.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZmsGfle1aZQauzRb@kbusch-mbp.dhcp.thefacebook.com>

On Thu, Jun 13, 2024 at 08:47:26AM -0600, Keith Busch wrote:
> On Thu, Jun 13, 2024 at 07:43:35AM -0700, Paul E. McKenney wrote:
> > On Thu, Jun 13, 2024 at 08:36:44AM -0600, Keith Busch wrote:
> > > On Thu, Jun 13, 2024 at 07:11:52PM +0530, Nilay Shroff wrote:
> > > > On 6/13/24 18:26, Keith Busch wrote:
> > > > > But that's not the problem for the rcu case. It's the last line that's
> > > > > the problem:
> > > > > 
> > > > >  	list->prev->next = list;
> > > > > 
> > > > > We can't change forward pointers for any element being detached from
> > > > > @head because a reader iterating the list may see that new pointer value
> > > > > and end up in the wrong list, breaking iteration. A synchronize rcu
> > > > > needs to happen before forward pointers can be mucked with, so it still
> > > > > needs to be done in two steps. Oh bother...
> > > > 
> > > > Agree and probably we may break it down using this API:
> > > > static inline void list_cut_rcu(struct list_head *list,
> > > >  		struct list_head *head, struct list_head *entry, 
> > > > 		void (*sync)(void))
> > > > {
> > > >  	list->next = entry;
> > > >  	list->prev = head->prev;
> > > > 	__list_del(entry->prev, head);
> > > > 	sync();
> > > >  	entry->prev = list;
> > > >  	list->prev->next = list;
> > > > }
> > > 
> > > Yes, that's the pattern, but I think we need an _srcu() variant: the
> > > "sync" callback needs to know the srcu_struct.
> > 
> > Just make a helper function like this:
> > 
> > 	static void my_synchronize_srcu(void)
> > 	{
> > 		synchronize_srcu(&my_srcu_struct);
> > 	}
> > 
> > Or am I missing something subtle here?
> 
> That would work if we had a global srcu, but the intended usage
> dynamically allocates one per device the driver is attached to, so a
> void callback doesn't know which one to sync.

Ah, good point!  I suppose that a further suggestion to just JIT the
needed function would not be well-received?  ;-)

I cannot resist suggesting placing a pointer to the srcu_struct in
the task structure.  /me runs...

Perhaps somewhat more constructively, my usual question:  Is it really
necessary to have per-driver SRCU here?  What would break if there was
a global srcu_struct that applied to all drivers?

						Thanx, Paul