Received-SPF: pass (google.com: domain of linux-kernel+bounces-87134-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Thu, 29 Feb 2024 09:00:58 -0800 (PST)
From: "Christoph Lameter (Ampere)" <cl@linux.com>
To: Shijie Huang <shijie@amperemail.onmicrosoft.com>
cc: Eric Dumazet <edumazet@google.com>, 
    Huang Shijie <shijie@os.amperecomputing.com>, kuba@kernel.org, 
    patches@amperecomputing.com, davem@davemloft.net, horms@kernel.org, 
    ast@kernel.org, dhowells@redhat.com, linyunsheng@huawei.com, 
    aleksander.lobakin@intel.com, linux-kernel@vger.kernel.org, 
    netdev@vger.kernel.org, cl@os.amperecomputing.com
Subject: Re: [PATCH v2] net: skbuff: set FLAG_SKB_NO_MERGE for
 skbuff_fclone_cache
In-Reply-To: <018b5652-8006-471d-94d0-d230e4aeef6d@amperemail.onmicrosoft.com>
Message-ID: <b7825865-c368-1b72-3751-f1928443db32@linux.com>
References: <CANn89iJoHDzfYfhcwVvR4m7DiVG-UfFNqm+D1WD-2wjOttk6ew@mail.gmail.com> <20240227062833.7404-1-shijie@os.amperecomputing.com> <CANn89iL2a2534d8QU9Xt6Gjm8M1+wWH03+YPdjSPQCq_Q4ZGxw@mail.gmail.com>
 <018b5652-8006-471d-94d0-d230e4aeef6d@amperemail.onmicrosoft.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII; format=flowed

On Wed, 28 Feb 2024, Shijie Huang wrote:

>> 
>> Using SLAB_NO_MERGE does not help, I am still seeing wrong allocations
>> on a dual socket
>> host with plenty of available memory.
>> (either sk_buff or skb->head being allocated on the other node).
>
> Do you mean you still can see the wrong fclone after using SLAB_NO_MERGE?
>
> If so, I guess there is bug in the slub.

Mergin has nothing to do with memory locality.

>> fclones might be allocated from a cpu running on node A, and freed
>> from a cpu running on node B.
>> Maybe SLUB is not properly handling this case ?
>
> Maybe.

Basic functionality is broken??? Really?

>> I think we need help from mm/slub experts, instead of trying to 'fix'
>> networking stacks.
>
> @Christopher
>
> Any idea about this?


If you want to force a local allocation then use GFP_THISNODE as a flag.

If you do not specify a node or GFP_THISNODE then the slub allocator will 
opportunistically allocate sporadically from other nodes to avoid 
fragmentation of slabs. The page allocator also will sporadically go off 
node in order to avoid reclaim. The page allocator may go off node 
extensively if there is a imbalance of allocation between node. The page 
allocator has knobs to tune off node vs reclaim options. Doing more 
reclaim will slow things down but give you local data.