Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Message-ID: <f4739117-ac81-4831-820b-60b496c7dd61@linux.dev>
Date:   Tue, 24 Oct 2023 19:03:10 +0800
MIME-Version: 1.0
Subject: Re: [RFC PATCH v2 0/6] slub: Delay freezing of CPU partial slabs
Content-Language: en-US
To:     "Christoph Lameter (Ampere)" <cl@gentwo.org>,
        Vlastimil Babka <vbabka@suse.cz>
Cc:     penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com,
        akpm@linux-foundation.org, roman.gushchin@linux.dev,
        42.hyeyoo@gmail.com, willy@infradead.org, pcc@google.com,
        tytso@mit.edu, maz@kernel.org, ruansy.fnst@fujitsu.com,
        vishal.moola@gmail.com, lrh2000@pku.edu.cn, hughd@google.com,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org,
        Chengming Zhou <zhouchengming@bytedance.com>
References: <20231021144317.3400916-1-chengming.zhou@linux.dev>
 <4134b039-fa99-70cd-3486-3d0c7632e4a3@suse.cz>
 <fcb9b7f0-fb21-cce8-d452-766a5cc73d4a@gentwo.org>
 <ba3d6ac7-6900-3e8d-46b5-8302ca61f8ef@suse.cz>
 <c6f12967-e7bc-bf36-9c6b-0111dea1f0de@gentwo.org>
From:   Chengming Zhou <chengming.zhou@linux.dev>
In-Reply-To: <c6f12967-e7bc-bf36-9c6b-0111dea1f0de@gentwo.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2023/10/24 05:05, Christoph Lameter (Ampere) wrote:
> On Mon, 23 Oct 2023, Vlastimil Babka wrote:
> 
>>> For much of the frozen handling we must be holding the node list lock
>>> anyways in order to add/remove from the list. So we already have a lock
>>> that could be used to protect flag operations.
>>
>> I can see the following differences between the traditional frozen bit and
>> the new flag:
>>
>> frozen bit advantage:
>> - __slab_free() on an already-frozen slab can ignore list operations and
>> list_lock completely
>>
>> frozen bit disadvantage:
>> - acquire_slab() trying to do cmpxchg_double() under list_lock (see commit
>> 9b1ea29bc0d7)
> 
> 
> Ok so a slab is frozen if either of those conditions are met. That gets a bit complicated to test for. Can we just get away with the slab_node_partial flag?
> 
> The advantage with the frozen state is that it can be changed with a cmpxchg together with some other values (list pointer, counter) that need updating at free and allocation.
> 
> But frozen updates are rarer so maybe its worth to completely drop the frozen bit. If both need to be updates then we would have two atomic ops. One is the cmpxchg and the other the operation on the page flag.
> 

This introduced page flag bit is using non-atomic operations, which is protected
by the node list_lock.

As for completely dropping the "frozen" bit, I find it hard because we have the
DEACTIVATE_BYPASS optimization in get_freelist(), which clear the "frozen" bit
without the synchronization of node list_lock. So __slab_free() still need to
rely on the "frozen" bit for CPU active slab.

This patch series mainly optimize the cmpxchg cost in moving partial slabs
between node partial list and CPU partial list, and alleviate the contention
of node list_lock meanwhile.

Thanks!