Received-SPF: pass (google.com: domain of linux-kernel+bounces-213349-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <c475f0d8-3bc9-4d65-8fce-586f4b75b4fc@linux.ibm.com>
Date: Thu, 13 Jun 2024 19:11:52 +0530
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] list: introduce a new cutting helper
To: Keith Busch <kbusch@kernel.org>
Cc: Keith Busch <kbusch@meta.com>, linux-nvme@lists.infradead.org,
        linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
        hch@lst.de, sagi@grimberg.me, paulmck@kernel.org, davidgow@google.com,
        akpm@linux-foundation.org, venkat88@linux.vnet.ibm.com
References: <20240612155135.3060667-1-kbusch@meta.com>
 <f0e4c51c-8227-4f5c-876f-38fbb4a0e1bf@linux.ibm.com>
 <ZmrscxG51gFRDVlM@kbusch-mbp>
Content-Language: en-US
From: Nilay Shroff <nilay@linux.ibm.com>
In-Reply-To: <ZmrscxG51gFRDVlM@kbusch-mbp>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit


On 6/13/24 18:26, Keith Busch wrote:
> On Thu, Jun 13, 2024 at 10:26:11AM +0530, Nilay Shroff wrote:
>> On 6/12/24 21:21, Keith Busch wrote:
>>> +static inline void list_cut(struct list_head *list,
>>> +		struct list_head *head, struct list_head *entry)
>>> +{
>>> +	list->next = entry;
>>> +	list->prev = head->prev;
>>> +	head->prev = entry->prev;
>>> +	entry->prev->next = head;
>>> +	entry->prev = list;
>>> +	list->prev->next = list;
>>> +}
>> I am wondering whether we really need the _rcu version of list_cut here?
>> I think that @head could point to an _rcu protected list and that's true 
>> for this patch. So there might be concurrent readers accessing @head using
>> _rcu list-traversal primitives, such as list_for_each_entry_rcu().
>>
>> An _rcu version of list_cut():
>>
>> static inline void list_cut_rcu(struct list_head *list,
>> 		struct list_head *head, struct list_head *entry)
>> {
>> 	list->next = entry;
>> 	list->prev = head->prev;
>> 	head->prev = entry->prev;
>> 	rcu_assign_pointer(list_next_rcu(entry->prev), head);
>> 	entry->prev = list;
>> 	list->prev->next = list;
>> }
> 
> I was initially thinking similiar, but this is really just doing a
> "list_del", and the rcu version calls the same generic __list_del()
> helper. To make this more clear, we could change
> 
> 	head->prev = entry->prev;
> 	entry->prev->next = head;
> 
> To just this:
> 
> 	__list_del(entry->prev, head);
> 
> And that also gets the "WRITE_ONCE" usage right.
Yeah this sounds reasonable.

> 
> But that's not the problem for the rcu case. It's the last line that's
> the problem:
> 
>  	list->prev->next = list;
> 
> We can't change forward pointers for any element being detached from
> @head because a reader iterating the list may see that new pointer value
> and end up in the wrong list, breaking iteration. A synchronize rcu
> needs to happen before forward pointers can be mucked with, so it still
> needs to be done in two steps. Oh bother...

Agree and probably we may break it down using this API:
static inline void list_cut_rcu(struct list_head *list,
 		struct list_head *head, struct list_head *entry, 
		void (*sync)(void))
{
 	list->next = entry;
 	list->prev = head->prev;
	__list_del(entry->prev, head);
	sync();
 	entry->prev = list;
 	list->prev->next = list;
}

Thanks,
--Nilay