Received-SPF: pass (google.com: domain of linux-kernel+bounces-213242-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Thu, 13 Jun 2024 06:56:19 -0600
From: Keith Busch <kbusch@kernel.org>
To: Nilay Shroff <nilay@linux.ibm.com>
Cc: Keith Busch <kbusch@meta.com>, linux-nvme@lists.infradead.org,
	linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	hch@lst.de, sagi@grimberg.me, paulmck@kernel.org,
	davidgow@google.com, akpm@linux-foundation.org,
	venkat88@linux.vnet.ibm.com
Subject: Re: [PATCH 1/2] list: introduce a new cutting helper
Message-ID: <ZmrscxG51gFRDVlM@kbusch-mbp>
References: <20240612155135.3060667-1-kbusch@meta.com>
 <f0e4c51c-8227-4f5c-876f-38fbb4a0e1bf@linux.ibm.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <f0e4c51c-8227-4f5c-876f-38fbb4a0e1bf@linux.ibm.com>

On Thu, Jun 13, 2024 at 10:26:11AM +0530, Nilay Shroff wrote:
> On 6/12/24 21:21, Keith Busch wrote:
> > +static inline void list_cut(struct list_head *list,
> > +		struct list_head *head, struct list_head *entry)
> > +{
> > +	list->next = entry;
> > +	list->prev = head->prev;
> > +	head->prev = entry->prev;
> > +	entry->prev->next = head;
> > +	entry->prev = list;
> > +	list->prev->next = list;
> > +}
> I am wondering whether we really need the _rcu version of list_cut here?
> I think that @head could point to an _rcu protected list and that's true 
> for this patch. So there might be concurrent readers accessing @head using
> _rcu list-traversal primitives, such as list_for_each_entry_rcu().
> 
> An _rcu version of list_cut():
> 
> static inline void list_cut_rcu(struct list_head *list,
> 		struct list_head *head, struct list_head *entry)
> {
> 	list->next = entry;
> 	list->prev = head->prev;
> 	head->prev = entry->prev;
> 	rcu_assign_pointer(list_next_rcu(entry->prev), head);
> 	entry->prev = list;
> 	list->prev->next = list;
> }

I was initially thinking similiar, but this is really just doing a
"list_del", and the rcu version calls the same generic __list_del()
helper. To make this more clear, we could change

	head->prev = entry->prev;
	entry->prev->next = head;

To just this:

	__list_del(entry->prev, head);

And that also gets the "WRITE_ONCE" usage right.

But that's not the problem for the rcu case. It's the last line that's
the problem:

 	list->prev->next = list;

We can't change forward pointers for any element being detached from
@head because a reader iterating the list may see that new pointer value
and end up in the wrong list, breaking iteration. A synchronize rcu
needs to happen before forward pointers can be mucked with, so it still
needs to be done in two steps. Oh bother...