Received-SPF: pass (google.com: domain of linux-kernel+bounces-81135-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <de696b4c-bda5-4ab2-a0c0-490b4eba8163@intel.com>
Date: Mon, 26 Feb 2024 11:10:07 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] net: skbuff: allocate the fclone in the current NUMA node
To: Huang Shijie <shijie@os.amperecomputing.com>
CC: <kuba@kernel.org>, <patches@amperecomputing.com>, <davem@davemloft.net>,
	<horms@kernel.org>, <edumazet@google.com>, <ast@kernel.org>,
	<dhowells@redhat.com>, <linyunsheng@huawei.com>,
	<linux-kernel@vger.kernel.org>, <netdev@vger.kernel.org>,
	<cl@os.amperecomputing.com>
References: <20240220021804.9541-1-shijie@os.amperecomputing.com>
From: Alexander Lobakin <aleksander.lobakin@intel.com>
Content-Language: en-US
In-Reply-To: <20240220021804.9541-1-shijie@os.amperecomputing.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?a1FncGdkQXlqSFdJaTF5cWxFZXQ5Z3RCWFlhdzhaMVRCb1k5MmhtRURpVUFl?=
 =?utf-8?B?TXhxWmxoWkdNVFpzblpWUFdxQzBCSXdBdjF0Ynk5amx3V2hwMmxiOWZTVUEz?=
 =?utf-8?B?cVlDcUMwbVBWN3kybXFRK3FhZUhYZnFzYlZYTkRBK1VKaVB4KytEUnd2S0s4?=
 =?utf-8?B?L04zbTVGL0xkMXhWcTllTVNHMVVvMGp4WEplYkhHbFcvajFsSnF1eUN0cXl0?=
 =?utf-8?B?NUd6dHlqRng5SjFXM0NvMnR5bkF5WkNHQXJzeEwrYmpKS0ZBbU9XaXVuZkhM?=
 =?utf-8?B?NnZKVThqZnVWdmJWcG5PdDRpUUFkalZKTWJYVFA1UFAwUXFJTmpqRGZTeTZm?=
 =?utf-8?B?dEtlSXZYcnBDWFNidFdMS1VpVHYyTmxLNlZsRHpXQitVbzQxZmc3dnpZTU03?=
 =?utf-8?B?by8yaHFaWG8rMWZ5QVd0QVlsdEs0OEF2RHFEQ3RzR1JTVkJISGtMZC9Wd3lp?=
 =?utf-8?B?TWVVSzFVR0JzT3RZQ3cxTVlxUkZOWjBnWFRWL003UVlYWE10Wnl2cTZjUHBv?=
 =?utf-8?B?RUQ0M1VNcjJMZ0U5OFNpOTBqMmxhL1RXa29XamJwcWp6eFBRa203dVZiTCtu?=
 =?utf-8?B?NGZUajZCMUZTaWdRc2lBbjdKc245bkJuVlhtTENEWU1IMG9ZY1VtUHZ1Wm9U?=
 =?utf-8?B?YnNiNWFpL2NEcTg0NUtXRUY1S2dNOVBlb0VvdjNkMlFVZTdBOVM5TWlRMXB4?=
 =?utf-8?B?aDlzT3ZRSTZSS2hER20wemdVcXVUcVNGWG9wUjhKQkthSnlRYlM2TTNKUDN6?=
 =?utf-8?B?ZTNBMU8vUDB1dTczdFdSeGFKbXBNQWFDa1F4b2MvM0FBdGVZS0thbXJYUHlF?=
 =?utf-8?B?b3BzUEduNGJaaU16djV4LzlEdUltc2F0S2NWTDVXdDFwWG91cFJ3OHQ0MXVS?=
 =?utf-8?B?YjhpRHRmb1Z6MGQ5OW40a2tHV09qTVJ1ZzdydzN1MTAzWjNhamI3NkVwU2lO?=
 =?utf-8?B?ektidjBFM0NqaDc3NEVTbGMyQ1RnK2huN1RBZ3ZDLzZicThCWHBYaHZqdXZG?=
 =?utf-8?B?cFpLai9CdXhzSjdIeFRtbVF0Q1pyaWxnNmpWUmM1dU90bmxFbnozdStMUUdo?=
 =?utf-8?B?SGUrRkI3dUF6Y2d1NUxmY2J3blJCb1Q3UkJWZkFsNVNmOHBKRTFKdzN2bzdH?=
 =?utf-8?B?SXl4ckRHU254V1VJSmdnOUlrb3J5SFRSUnJPWTVwQUpJM2NwT0pGdEc2WEt2?=
 =?utf-8?B?TDdEL3pJRnBySHlMcUZCbnFiZnB1TFdSZklSZEVvQVpQdVM5cWJjZ1FsR21G?=
 =?utf-8?B?ZVhSV1B2bTIzZkdzVlE2VWJuVlJiWjR4byt1SVl1ZTlSaVZHVkhVZmNvckdu?=
 =?utf-8?B?bU5HeERHRitmV1BtTEp0VXpBRWIvYzE2ZUZycWxnbGovNENXL2kyV1c2YkZ0?=
 =?utf-8?B?dEtEbG54YlZBN3FZN2FjSFNXdEl6eXpFRmpFd2VqTWRhTjBqbStyYUV4aVhr?=
 =?utf-8?B?QnB3QVA1eFdNV1Y3V0lveWgxdDNZVjZRRG40TWV2Qno3NEpvWE5Kc1lZcjB6?=
 =?utf-8?B?dVBZQXgzVWRsUUliZ2ZmZFdwMnZQb3NnSmY2YzByeHBKZGxHMTErUXBWMEhT?=
 =?utf-8?B?dUs0SzlQTWg4QU51Y2xjaXA2dWFUSHAxREFuZ3B0VmpOVEloSlYxT3JUNnl6?=
 =?utf-8?B?L1VBTmVTM1FzMGc2VGZwMTdpdGJZR28rK0gxUFdHRDV2aVplV2lzV0tMbWR3?=
 =?utf-8?B?YUVBeHNWczFMYmRZVGpaNWl3YUFpN0gyM2RKMFhXeGZNZ1plUnRCNTU5ckE2?=
 =?utf-8?B?YjN1bDJmdmEwN291bUpCR1l3U1ErdVUyeFBSK0E4cVo5RjFCNEx2VE41bmhE?=
 =?utf-8?B?WjBnaWNDdGVzRmQxaEJqMHltOUNnY3ZxVXFnTStTb1JmQ3VONEU2QzVwb2hK?=
 =?utf-8?B?MUtQR2dsUlZCN2xoclVnRHUrSmJ5SWdnVmxseWdjUXlhdmJCTDBaNklaQTNH?=
 =?utf-8?B?Q3ZoaGNEa2ZBSTNUMURRVnA3L2YzOGpXRWVMRVN2L3VYdXp6ODN2QnVYZ3JI?=
 =?utf-8?B?WFRzVURWMW1Nb2w0NkwvY2xEYjhUSnI5aGxDR2N6MEVYUGljNFFHTUJ6MGk5?=
 =?utf-8?B?bDlySy94MnpndS9Qc09DczZTbmtUWXpnelFRc0dYeDdRRGFWV2NDSjRFR005?=
 =?utf-8?B?VFNlRjgraSttVkJDV2VPNEg1OU5NUXFFbEtObmFxV1ZlTm5GQnRrVGNBN1pU?=
 =?utf-8?B?cUE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 903b0b50-a2c4-42a6-70a1-08dc36b32040
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB8718.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 10:10:15.0119
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RIyKpt+OepV8Ed0rks2Vxi6pIQwmUo/20PK69Kgg4Kppw5hHT/WaoU0Ds6P1RChhrqKtbtVhDXcAiNyO58i+7s+AxqSiUJ6Mmbf59eGunbk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR11MB7251
X-OriginatorOrg: intel.com

From: Huang Shijie <shijie@os.amperecomputing.com>
Date: Tue, 20 Feb 2024 10:18:04 +0800

> The current code passes NUMA_NO_NODE to __alloc_skb(), we found
> it may creates fclone SKB in remote NUMA node.
> 
> So use numa_node_id() to limit the allocation to current NUMA node.
> 
> Signed-off-by: Huang Shijie <shijie@os.amperecomputing.com>
> ---
>  include/linux/skbuff.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
> index 2dde34c29203..ebc42b2604ad 100644
> --- a/include/linux/skbuff.h
> +++ b/include/linux/skbuff.h
> @@ -1343,7 +1343,7 @@ static inline bool skb_fclone_busy(const struct sock *sk,
>  static inline struct sk_buff *alloc_skb_fclone(unsigned int size,
>  					       gfp_t priority)
>  {
> -	return __alloc_skb(size, priority, SKB_ALLOC_FCLONE, NUMA_NO_NODE);
> +	return __alloc_skb(size, priority, SKB_ALLOC_FCLONE, numa_node_id());

Because it tries to defragment the memory and pick an optimal node.

__alloc_skb() and skb clones aren't anyway something very hotpathish, do
you have any particular perf numbers and/or usecases where %NUMA_NO_NODE
really hurts?

>  }
>  
>  struct sk_buff *skb_morph(struct sk_buff *dst, struct sk_buff *src);

Thanks,
Olek