Received-SPF: pass (google.com: domain of linux-kernel+bounces-87221-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Thu, 29 Feb 2024 09:57:43 -0800 (PST)
From: "Christoph Lameter (Ampere)" <cl@linux.com>
To: Eric Dumazet <edumazet@google.com>
cc: Shijie Huang <shijie@amperemail.onmicrosoft.com>, 
    Huang Shijie <shijie@os.amperecomputing.com>, kuba@kernel.org, 
    patches@amperecomputing.com, davem@davemloft.net, horms@kernel.org, 
    ast@kernel.org, dhowells@redhat.com, linyunsheng@huawei.com, 
    aleksander.lobakin@intel.com, linux-kernel@vger.kernel.org, 
    netdev@vger.kernel.org
Subject: Re: [PATCH v2] net: skbuff: set FLAG_SKB_NO_MERGE for
 skbuff_fclone_cache
In-Reply-To: <CANn89iJpZ6udACMC9EF=zgYJU5rqLFiTuYJRf1UNA3UKu7CxJg@mail.gmail.com>
Message-ID: <1e72ab23-8161-091e-dc9e-9ecfe84a02df@linux.com>
References: <CANn89iJoHDzfYfhcwVvR4m7DiVG-UfFNqm+D1WD-2wjOttk6ew@mail.gmail.com> <20240227062833.7404-1-shijie@os.amperecomputing.com> <CANn89iL2a2534d8QU9Xt6Gjm8M1+wWH03+YPdjSPQCq_Q4ZGxw@mail.gmail.com> <018b5652-8006-471d-94d0-d230e4aeef6d@amperemail.onmicrosoft.com>
 <b7825865-c368-1b72-3751-f1928443db32@linux.com> <CANn89iJpZ6udACMC9EF=zgYJU5rqLFiTuYJRf1UNA3UKu7CxJg@mail.gmail.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII; format=flowed

On Thu, 29 Feb 2024, Eric Dumazet wrote:

>> If you do not specify a node or GFP_THISNODE then the slub allocator will
>> opportunistically allocate sporadically from other nodes to avoid
>> fragmentation of slabs. The page allocator also will sporadically go off
>> node in order to avoid reclaim. The page allocator may go off node
>> extensively if there is a imbalance of allocation between node. The page
>> allocator has knobs to tune off node vs reclaim options. Doing more
>> reclaim will slow things down but give you local data.
>
> Maybe, maybe not.
>
> Going back to CONFIG_SLAB=y removes all mismatches, without having to
> use GFP_THISNODE at all,
> on hosts with plenty of available memory on all nodes.


Slab uses GFPTHISNODE by default and does not respect the memory policies 
etc set for pages. As such it will causes additional overhead through 
reclaim passses etc and memory policies will not be applied on a per page 
level (as specd) but in its own layer on a per object basis. It causes 
additional fragmentation.

> I think that is some kind of evidence that something is broken in SLUB land.

That is one of the reasons that SLAB was removed.

Slub defragmentation can be disabled by either GFP_THISNODE or tuning the 
remote_claim knob in /sys/kernel/slab/<slabname>